Module datafusion::datasource
source · Expand description
DataFusion data sources: TableProvider
and ListingTable
Re-exports
pub use self::default_table_source::provider_as_source;
pub use self::default_table_source::source_as_provider;
pub use self::default_table_source::DefaultTableSource;
pub use self::memory::MemTable;
pub use self::provider::TableProvider;
pub use self::view::ViewTable;
Modules
- This module contains code for reading Avro data into
RecordBatch
es - Default TableSource implementation used in DataFusion physical plans
EmptyTable
useful for testing.- Module containing helper methods for the various file formats
- A table that uses the
ObjectStore
listing capability to get the list of files to process. - Factory for creating ListingTables with default options
MemTable
for queryingVec<RecordBatch>
by DataFusion.- ObjectStoreRegistry holds all the object stores at Runtime with a scheme for each store. This allows the user to extend DataFusion with different storage systems such as S3 or HDFS and query data inside these systems.
- Execution plans that read file formats
- Data source traits
- A simplified
TableProvider
for streaming partitioned datasets - View data source which uses a LogicalPlan as it’s input.
Enums
- Indicates the type of this table for metadata/catalog purposes.
Functions
- Get all files as well as the file level summary statistics (no statistic for partition columns). If the optional
limit
is provided, includes only sufficient files. Needed to read up tolimit
number of rows.