Crate datafusion_datasource_parquet 

DataFusion Parquet Reader: ParquetSource

Re-exports

pub use access_plan::ParquetAccessPlan;

pub use access_plan::ParquetRowSelection;

pub use access_plan::RowGroupAccess;

pub use file_format::*;

Modules

access_plan

file_format

ParquetFormat: Parquet FileFormat abstractions

metadata

DFParquetMetadata for fetching Parquet file metadata, statistics and schema information.

source

ParquetSource implementation for reading parquet files

Structs

BloomFilterStatistics

In memory Parquet Split Block Bloom Filters (SBBF).

CachedParquetFileReaderFactory

Implementation of ParquetFileReaderFactory supporting the caching of footer and page metadata. Reads and updates the FileMetadataCache with the ParquetMetaData data.

CachedParquetMetaData

Wrapper to implement FileMetadata for ParquetMetaData.

DefaultParquetFileReaderFactory

Default implementation of ParquetFileReaderFactory

Int96Coercer

Builder for coercing INT96-originated Timestamp columns in file_schema to a specific TimeUnit, optionally attaching a timezone.

PagePruningAccessPlanFilter

Filters a ParquetAccessPlan based on the Parquet PageIndex, if present

ParquetFileMetrics

Stores metrics about the parquet execution for a particular parquet file.

ParquetFileReader

Implements AsyncFileReader for a parquet file in object storage.

ParquetSink

Implements DataSink for writing to a parquet file.

RowGroupAccessPlanFilter

Reduces the ParquetAccessPlan based on row group level metadata.

Enums

ParquetVirtualColumn

A parquet virtual column validated to have a supported arrow extension type.

Traits

ParquetFileReaderFactory

Interface for reading Apache Parquet files.

Functions

apply_file_schema_type_coercions

Apply necessary schema type coercions to make file schema match table schema.

build_row_filter

Build a RowFilter from the given predicate expression if possible.

can_expr_be_pushed_down_with_schemas

Checks if a predicate expression can be pushed down to the parquet decoder.

coerce_int96_to_resolutionDeprecated

Coerces the file schema’s Timestamps to the provided TimeUnit if the Parquet schema contains INT96.

plan_to_parquet

Executes a query and writes the results to a partitioned Parquet file.

transform_binary_to_string

Transform a schema so that any binary types are strings

transform_schema_to_view

Transform a schema to use view types for Utf8 and Binary