Module datafusion::physical_optimizer
source · [−]Expand description
This module contains a query optimizer that operates against a physical plan and applies rules to a physical plan, such as “Repartition”.
Re-exports
pub use optimizer::PhysicalOptimizerRule;
Modules
Utilizing exact statistics from sources to avoid scanning data
CoalesceBatches optimizer that groups batches together rows
in bigger batches to avoid overhead with small batches
Utilizing exact statistics from sources to avoid scanning data
AddCoalescePartitionsExec adds CoalescePartitionsExec to plans
with more than one partition, to coalesce them into one partition
when the node needs a single partition
Physical optimizer traits
This module contains code to prune “containers” of row groups
based on statistics prior to execution. This can lead to
significant performance improvements by avoiding the need
to evaluate a plan on entire containers (e.g. an entire file)
Repartition optimizer that introduces repartition nodes to increase the level of parallelism available