Crate pruning 

Structs

FilePruner

Prune based on file-level statistics.

PredicateRewriter

Rewrite a predicate expression in terms of statistics (min/max/null_counts) for use as a PruningPredicate.

PruningPredicate

Used to prove that arbitrary predicates (boolean expression) can not possibly evaluate to true given information about a column provided by PruningStatistics.

PruningPredicateBuilder

Builder for a PruningPredicate. Groups optional configuration — IN (...) rewrite cap, error counter — so future additions do not churn the top-level API.

RequiredColumns

Describes which columns statistics are necessary to evaluate a PruningPredicate.

Constants

MAX_IN_LIST_SIZE

Default maximum number of entries in an IN (...) list that will be rewritten into a chain of per-value min/max checks by build_predicate_expression. Callers threading a PredicateRewriter can override this via PredicateRewriter::with_max_in_list_size, and query engines can wire it from the datafusion.execution.parquet.max_in_list_size config option.

Traits

PruningStatistics

A source of runtime statistical information to PruningPredicates.

UnhandledPredicateHook

Rewrites predicates that PredicateRewriter can not handle, e.g. certain complex expressions or predicates that reference columns that are not in the schema.

Functions

build_pruning_predicate

Build a pruning predicate from an optional predicate expression. If the predicate is None or the predicate cannot be converted to a pruning predicate, return None. If there is an error creating the pruning predicate it is recorded by incrementing the predicate_creation_errors counter.