Module datafusion::datasource
source · Expand description
DataFusion data sources
Re-exports
pub use self::datasource::TableProvider;
pub use self::default_table_source::provider_as_source;
pub use self::default_table_source::source_as_provider;
pub use self::default_table_source::DefaultTableSource;
pub use self::memory::MemTable;
pub use self::view::ViewTable;
Modules
Data source traits
Default TableSource implementation used in DataFusion physical plans
An empty plan that is usefull for testing and generating plans without mapping them to actual data.
Module containing helper methods for the various file formats
A table that uses the
ObjectStore
listing capability
to get the list of files to process.Factory for creating ListingTables with default options
In-memory data source for presenting a Vec as a data source that can be
queried by DataFusion. This allows data to be pre-loaded into memory and then
repeatedly queried without incurring additional file I/O overhead.
ObjectStoreRegistry holds all the object stores at Runtime with a scheme for each store.
This allows the user to extend DataFusion with different storage systems such as S3 or HDFS
and query data inside these systems.
A simplified
TableProvider
for streaming partitioned datasetsView data source which uses a LogicalPlan as it’s input.
Enums
Indicates the type of this table for metadata/catalog purposes.
Functions
Get all files as well as the file level summary statistics (no statistic for partition columns).
If the optional
limit
is provided, includes only sufficient files.
Needed to read up to limit
number of rows.