Crate datafusion_datasource_parquet

Expand description

DataFusion Parquet Reader: ParquetSource

Re-exports§

access_plan
file_format: ParquetFormat: Parquet FileFormat abstractions
metadata: DFParquetMetadata for fetching Parquet file metadata, statistics and schema information.
source: ParquetSource implementation for reading parquet files

CachedParquetFileReader: Implements AsyncFileReader for a Parquet file in object storage. Reads the file metadata from the FileMetadataCache, if available, otherwise reads it directly from the file and then updates the cache.
CachedParquetFileReaderFactory: Implementation of ParquetFileReaderFactory supporting the caching of footer and page metadata. Reads and updates the FileMetadataCache with the ParquetMetaData data. This reader always loads the entire metadata (including page index, unless the file is encrypted), even if not required by the current query, to ensure it is always available for those that need it.
CachedParquetMetaData: Wrapper to implement FileMetadata for ParquetMetaData.
DefaultParquetFileReaderFactory: Default implementation of ParquetFileReaderFactory
Int96Coercer: Builder for coercing INT96-originated Timestamp columns in file_schema to a specific TimeUnit, optionally attaching a timezone.
PagePruningAccessPlanFilter: Filters a ParquetAccessPlan based on the Parquet PageIndex, if present
ParquetFileMetrics: Stores metrics about the parquet execution for a particular parquet file.
ParquetFileReader: Implements AsyncFileReader for a parquet file in object storage.
ParquetSink: Implements DataSink for writing to a parquet file.
RowGroupAccessPlanFilter: Reduces the ParquetAccessPlan based on row group level metadata.

apply_file_schema_type_coercions: Apply necessary schema type coercions to make file schema match table schema.
build_row_filter: Build a RowFilter from the given predicate expression if possible.
can_expr_be_pushed_down_with_schemas: Checks if a predicate expression can be pushed down to the parquet decoder.
coerce_file_schema_to_string_typeDeprecated: If the table schema uses a string type, coerce the file schema to use a string type.
coerce_file_schema_to_view_typeDeprecated: Coerces the file schema if the table schema uses a view type.
coerce_int96_to_resolutionDeprecated: Coerces the file schema’s Timestamps to the provided TimeUnit if the Parquet schema contains INT96.
plan_to_parquet: Executes a query and writes the results to a partitioned Parquet file.
transform_binary_to_string: Transform a schema so that any binary types are strings
transform_schema_to_view: Transform a schema to use view types for Utf8 and Binary