Crate datafusion_datasource_parquet 

Re-exports

pub use access_plan::ParquetAccessPlan;

pub use access_plan::RowGroupAccess;

pub use file_format::*;

Modules

access_plan

file_format

ParquetFormat: Parquet FileFormat abstractions

metadata

DFParquetMetadata for fetching Parquet file metadata, statistics and schema information.

source

ParquetSource implementation for reading parquet files

Structs

CachedParquetFileReader

Implements AsyncFileReader for a Parquet file in object storage. Reads the file metadata from the FileMetadataCache, if available, otherwise reads it directly from the file and then updates the cache.

CachedParquetFileReaderFactory

Implementation of ParquetFileReaderFactory supporting the caching of footer and page metadata. Reads and updates the FileMetadataCache with the ParquetMetaData data. This reader always loads the entire metadata (including page index, unless the file is encrypted), even if not required by the current query, to ensure it is always available for those that need it.

CachedParquetMetaData

Wrapper to implement FileMetadata for ParquetMetaData.

DefaultParquetFileReaderFactory

Default implementation of ParquetFileReaderFactory

PagePruningAccessPlanFilter

Filters a ParquetAccessPlan based on the Parquet PageIndex, if present

ParquetFileMetrics

Stores metrics about the parquet execution for a particular parquet file.

ParquetFileReader

Implements AsyncFileReader for a parquet file in object storage.

RowGroupAccessPlanFilter

Reduces the ParquetAccessPlan based on row group level metadata.

Traits

ParquetFileReaderFactory

Interface for reading parquet files.

Functions

build_row_filter

Build a RowFilter from the given predicate expression if possible.

can_expr_be_pushed_down_with_schemas

Checks if a predicate expression can be pushed down to the parquet decoder.

plan_to_parquet

Executes a query and writes the results to a partitioned Parquet file.