Module datafusion::physical_optimizer

This module contains a query optimizer that operates against a physical plan and applies rules to a physical plan, such as “Repartition”.

Re-exports

pub use optimizer::PhysicalOptimizerRule;

Modules

aggregate_statistics

Utilizing exact statistics from sources to avoid scanning data

coalesce_batches

CoalesceBatches optimizer that groups batches together rows in bigger batches to avoid overhead with small batches

hash_build_probe_order

Utilizing exact statistics from sources to avoid scanning data

merge_exec

AddCoalescePartitionsExec adds CoalescePartitionsExec to plans with more than one partition, to coalesce them into one partition when the node needs a single partition

optimizer

Physical optimizer traits

pruning

This module contains code to prune “containers” of row groups based on statistics prior to execution. This can lead to significant performance improvements by avoiding the need to evaluate a plan on entire containers (e.g. an entire file)

repartition

Repartition optimizer that introduces repartition nodes to increase the level of parallelism available