Struct datafusion::optimizer::filter_push_down::FilterPushDown

pub struct FilterPushDown {}

Filter Push Down optimizer rule pushes filter clauses down the plan

Introduction

A filter-commutative operation is an operation whose result of filter(op(data)) = op(filter(data)). An example of a filter-commutative operation is a projection; a counter-example is limit.

The filter-commutative property is column-specific. An aggregate grouped by A on SUM(B) can commute with a filter that depends on A only, but does not commute with a filter that depends on SUM(B).

This optimizer commutes filters with filter-commutative operations to push the filters the closest possible to the scans, re-writing the filter expressions by every projection that changes the filter’s expression.

Filter: #b Gt Int64(10) Projection: #a AS b

is optimized to

Projection: #a AS b Filter: #a Gt Int64(10) <— changed from #b to #a

This performs a single pass trought the plan. When it passes trought a filter, it stores that filter, and when it reaches a node that does not commute with it, it adds the filter to that place. When it passes through a projection, it re-writes the filter’s expression taking into accoun that projection. When multiple filters would have been written, it AND their expressions into a single expression.

Implementations

impl FilterPushDown

pub fn new() -> Self

Trait Implementations

impl OptimizerRule for FilterPushDown

fn name(&self) -> &str

A human readable name for this optimizer rule

fn optimize(&self, plan: &LogicalPlan) -> Result<LogicalPlan>

Rewrite plan to an optimized form