Module datafusion::physical_optimizer
source · [−]Expand description
This module contains a query optimizer that operates against a physical plan and applies rules to a physical plan, such as “Repartition”.
Modules
Utilizing exact statistics from sources to avoid scanning data
CoalesceBatches optimizer that groups batches together rows in bigger batches to avoid overhead with small batches
Utilizing exact statistics from sources to avoid scanning data
AddCoalescePartitionsExec adds CoalescePartitionsExec to plans with more than one partition, to coalesce them into one partition when the node needs a single partition
Physical optimizer traits
This module contains code to prune “containers” of row groups based on statistics prior to execution. This can lead to significant performance improvements by avoiding the need to evaluate a plan on entire containers (e.g. an entire file)
Repartition optimizer that introduces repartition nodes to increase the level of parallism available