vortex_datafusion/v2/mod.rs
1// SPDX-License-Identifier: Apache-2.0
2// SPDX-FileCopyrightText: Copyright the Vortex contributors
3
4//! Direct DataFusion integration for an existing Vortex
5//! [`DataSourceRef`].
6//!
7//! Use this module when some other part of the system has already selected the
8//! Vortex source to query and DataFusion only needs an adapter around it.
9//!
10//! Typical flow:
11//!
12//! 1. Build or obtain a Vortex [`DataSourceRef`].
13//! 2. Wrap it in [`VortexTable`] to register it with a [`SessionContext`], or
14//! build a [`VortexDataSource`] directly when constructing a
15//! [`DataSourceExec`].
16//! 3. Let DataFusion apply projection, filter, and limit pushdown through the
17//! resulting adapter.
18//!
19//! The two main types are:
20//!
21//! - [`VortexTable`], the higher-level
22//! [`TableProvider`] for `SessionContext::register_table`.
23//! - [`VortexDataSource`], the lower-level
24//! [`DataSource`] used when constructing physical plans directly.
25//!
26//! Compared with [`crate::VortexFormatFactory`], this module starts from an
27//! already-constructed Vortex source instead of asking DataFusion to discover
28//! `.vortex` files.
29//!
30//! [`DataSourceRef`]: vortex::scan::DataSourceRef
31//! [`SessionContext`]: https://docs.rs/datafusion/latest/datafusion/prelude/struct.SessionContext.html
32//! [`DataSourceExec`]: datafusion_datasource::source::DataSourceExec
33//! [`TableProvider`]: datafusion_catalog::TableProvider
34//! [`DataSource`]: datafusion_datasource::source::DataSource
35
36mod source;
37mod table;
38
39pub use source::VortexDataSource;
40pub use table::VortexTable;