Crate rudb_opt 

The rewrite passes, cardinality estimation, join ordering, predicate transfer and layout adaptation.

Rank 11 in the layer rule. See xtask/layers.toml and spec/18-package-layout.md.

Twenty five passes so far. spec/09-optimizer.md section 9.1 describes a sequence and PASSES is the start of it. Column pruning came first, because it is the pass whose absence is measured in gigabytes: a scan that reads 105 columns to answer a question about three is the whole of the difference on ClickBench, and the Parquet reader has been able to read a subset since M1 with nothing able to tell it which subset.

Modules

bounds

Reading a filter as tests a store’s minimum and maximum can answer.

cluster

When a grouped aggregate’s key is a column the table is stored in ascending order of, so a group is finished as soon as the key moves past it.

columns

Column pruning, which is the scan half of projection pushdown.

cte

Dropping a materialisation nothing reads.

delim

Taking the domain back out of a decorrelated existence test.

dense

When a grouped aggregate’s key is an integer column with no gaps worth speaking of, so the slot a row belongs in can be read rather than looked for.

dependent

Removes group keys whose value is determined by other group keys.

distinct

Turning COUNT(DISTINCT x) into a grouping, so that it runs on the machinery grouping already has.

eliminate

The three rewrites a verified relationship licenses, which all delete work rather than speed it up.

empty

Replacing a subtree that cannot produce a row with a relation that produces none.

estimate

How many rows a node produces, guessed.

explain

What EXPLAIN prints.

extremes

Answering MIN and MAX out of the zone maps instead of off the column.

filter

Filter pushdown.

fold

Constant folding and the simplifications that fall out of it.

fromkey

An aggregate that only reads the column it groups on is a projection over a count.

keys

Pushing the keys an outer query asks about into the aggregate that answers it.

late

Late materialisation: reading the wide columns after the limit rather than before it.

limit

Moving a limit below the projection above it.

link

Choosing the join that reads a link over the join that builds a hash table.

nonulls

Answering the null questions a store has already answered, on a column that has no nulls.

nulls

What a predicate says about a row that a join padded with nulls.

order

Choosing which order a run of inner joins runs in.

pass

What a rewrite is, and what it is given besides the plan.

presize

How large a grouped aggregate’s hash table should be before its first row arrives.

reorder

Which conjunct of a filter runs first.

semi

Semi joins: making them, and then moving them to where they are worth having.

sides

Which input of each join the executor gathers whole.

tables

Which tables an expression reads, and which ones an operator produces.

topn

Turning a limit over a sort into a top N.

unnest

Turns correlated subqueries into set-based relational operators.

Constants

RANK

The crate this rank belongs to, so that the layer check has something to read.

Statics

PASSES

The passes, in the order they run.

UPSTREAM

Every name SET disabled_optimizers accepts, which is every name DuckDB accepts.

Functions

optimize

Rewrites a bound plan into the plan that runs, with every pass on.

optimize_with

Rewrites a bound plan into the plan that runs, skipping the passes the context turned off.