Module file_format

ParquetFormat: Parquet FileFormat abstractions

Structs

ParquetFormat

The Apache Parquet FileFormat implementation

ParquetFormatFactory

Factory struct used to create ParquetFormat

ParquetSink

Implements DataSink for writing to a parquet file.

Functions

apply_file_schema_type_coercions

Apply necessary schema type coercions to make file schema match table schema.

coerce_file_schema_to_string_typeDeprecated

If the table schema uses a string type, coerce the file schema to use a string type.

coerce_file_schema_to_view_typeDeprecated

Coerces the file schema if the table schema uses a view type.

coerce_int96_to_resolution

Coerces the file schema’s Timestamps to the provided TimeUnit if Parquet schema contains INT96.

fetch_parquet_metadata

Fetches parquet metadata from ObjectStore for given object

fetch_statistics

Read and parse the statistics of the Parquet file at location path

statistics_from_parquet_meta_calc

Convert statistics in ParquetMetaData into Statistics using StatisticsConverter

transform_binary_to_string

Transform a schema so that any binary types are strings

transform_schema_to_view

Transform a schema to use view types for Utf8 and Binary