Struct deltalake::datafusion::optimizer::rewrite_disjunctive_predicate::RewriteDisjunctivePredicate

pub struct RewriteDisjunctivePredicate;

Optimizer pass that rewrites predicates of the form

(A = B AND <expr1>) OR (A = B AND <expr2>) OR ... (A = B AND <exprN>)

Into

(A = B) AND (<expr1> OR <expr2> OR ... <exprN> )

Predicates connected by OR typically not able to be broken down and distributed as well as those connected by AND.

The idea is to rewrite predicates into good_predicate1 AND good_predicate2 AND ... where good_predicate means the predicate has special support in the execution engine.

Equality join predicates (e.g. col1 = col2), or single column expressions (e.g. col = 5) are examples of predicates with special support.

TPCH Q19

This optimization is admittedly somewhat of a niche usecase. It’s main use is that it appears in TPCH Q19 and is required to avoid a CROSS JOIN.

Specifically, Q19 has a WHERE clause that looks like

where
  p_partkey = l_partkey
  and l_shipmode in (‘AIR’, ‘AIR REG’)
  and l_shipinstruct = ‘DELIVER IN PERSON’
  and (
    (
      and p_brand = ‘[BRAND1]’
      and p_container in ( ‘SM CASE’, ‘SM BOX’, ‘SM PACK’, ‘SM PKG’)
      and l_quantity >= [QUANTITY1] and l_quantity <= [QUANTITY1] + 10
      and p_size between 1 and 5
    )
    or
    (
      and p_brand = ‘[BRAND2]’
      and p_container in (‘MED BAG’, ‘MED BOX’, ‘MED PKG’, ‘MED PACK’)
      and l_quantity >= [QUANTITY2] and l_quantity <= [QUANTITY2] + 10
      and p_size between 1 and 10
    )
    or
    (
      and p_brand = ‘[BRAND3]’
      and p_container in ( ‘LG CASE’, ‘LG BOX’, ‘LG PACK’, ‘LG PKG’)
      and l_quantity >= [QUANTITY3] and l_quantity <= [QUANTITY3] + 10
      and p_size between 1 and 15
    )
)

Naively planning this query will result in a CROSS join with that single large OR filter. However, rewriting it using the rewrite in this pass results in a proper join predicate, p_partkey = l_partkey:

where
  p_partkey = l_partkey
  and l_shipmode in (‘AIR’, ‘AIR REG’)
  and l_shipinstruct = ‘DELIVER IN PERSON’
  and (
    (
      and p_brand = ‘[BRAND1]’
      and p_container in ( ‘SM CASE’, ‘SM BOX’, ‘SM PACK’, ‘SM PKG’)
      and l_quantity >= [QUANTITY1] and l_quantity <= [QUANTITY1] + 10
      and p_size between 1 and 5
    )
    or
    (
      and p_brand = ‘[BRAND2]’
      and p_container in (‘MED BAG’, ‘MED BOX’, ‘MED PKG’, ‘MED PACK’)
      and l_quantity >= [QUANTITY2] and l_quantity <= [QUANTITY2] + 10
      and p_size between 1 and 10
    )
    or
    (
      and p_brand = ‘[BRAND3]’
      and p_container in ( ‘LG CASE’, ‘LG BOX’, ‘LG PACK’, ‘LG PKG’)
      and l_quantity >= [QUANTITY3] and l_quantity <= [QUANTITY3] + 10
      and p_size between 1 and 15
    )
)

Implementations

impl RewriteDisjunctivePredicate

pub fn new() -> RewriteDisjunctivePredicate

Trait Implementations

impl Default for RewriteDisjunctivePredicate

fn default() -> RewriteDisjunctivePredicate

Returns the “default value” for a type.

impl OptimizerRule for RewriteDisjunctivePredicate

fn try_optimize( &self, plan: &LogicalPlan, _config: &dyn OptimizerConfig ) -> Result<Option<LogicalPlan>, DataFusionError>

Try and rewrite plan to an optimized form, returning None if the plan cannot be optimized by this rule.

fn name(&self) -> &str

A human readable name for this optimizer rule

fn apply_order(&self) -> Option<ApplyOrder>

How should the rule be applied by the optimizer? See comments on ApplyOrder for details.