Struct datafusion::config::ParquetOptions
source · pub struct ParquetOptions {
pub enable_page_index: bool,
pub pruning: bool,
pub skip_metadata: bool,
pub metadata_size_hint: Option<usize>,
pub pushdown_filters: bool,
pub reorder_filters: bool,
}
Expand description
Options related to reading of parquet files
Fields (Non-exhaustive)§
This struct is marked as non-exhaustive
Struct { .. }
syntax; cannot be matched against without a wildcard ..
; and struct update syntax will not work.enable_page_index: bool
If true, uses parquet data page level metadata (Page Index) statistics to reduce the number of rows decoded.
pruning: bool
If true, the parquet reader attempts to skip entire row groups based on the predicate in the query and the metadata (min/max values) stored in the parquet file
skip_metadata: bool
If true, the parquet reader skip the optional embedded metadata that may be in the file Schema. This setting can help avoid schema conflicts when querying multiple parquet files with schemas containing compatible types but different metadata
metadata_size_hint: Option<usize>
If specified, the parquet reader will try and fetch the last size_hint
bytes of the parquet file optimistically. If not specified, two reads are required:
One read to fetch the 8-byte parquet footer and
another to fetch the metadata length encoded in the footer
pushdown_filters: bool
If true, filter expressions are be applied during the parquet decoding operation to reduce the number of rows decoded
reorder_filters: bool
If true, filter expressions evaluated during the parquet decoding operation will be reordered heuristically to minimize the cost of evaluation. If false, the filters are applied in the same order as written in the query
Trait Implementations§
source§impl Clone for ParquetOptions
impl Clone for ParquetOptions
source§fn clone(&self) -> ParquetOptions
fn clone(&self) -> ParquetOptions
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read more