Function with_krishiv_optimizer_rules 

pub fn with_krishiv_optimizer_rules(
    builder: SessionStateBuilder,
) -> SessionStateBuilder

Build the DataFusion session config with a configurable parallelism level.

When target_partitions > 1, round-robin repartitioning is enabled so DataFusion can balance work across threads for hash-join build, aggregation spill, and parquet scan parallelism.

execution.batch_size is set from KRISHIV_BATCH_SIZE (default: 8192).

memory_limit_bytes, when Some, scales sort_spill_reservation_bytes down proportionally so a tight memory pool can still spill instead of failing outright because the reservation itself doesn’t fit. Pools at or above 4 * DEFAULT_SORT_SPILL_RESERVATION_BYTES (40MB) are unaffected — this only kicks in for genuinely memory-constrained deployments. Install Krishiv’s optimizer rules on a session-state builder.

A6 (review 2026-07-27). These rules used to be written out at each construction site, with a comment at one of them warning that “a rule installed on only one of them is indistinguishable from a rule that works until you hit the other path”. That warning was correct and the drift happened anyway: there is a third site — planning_session_context, the context the coordinator plans every distributed query on — and it carried none of these. The consequence was that two shipped performance fixes did not apply to the path being benchmarked:

• SpillableJoinSelection is what lets q18’s oversized hash-join build side become a spillable sort-merge join. Unregistered, q18 fails with Resources exhausted: HashJoinInput on every distributed run.
• SemiJoinReductionThroughAggregate is 88 % of q17’s runtime.
• CooperativeAmplifiers is what lets distributed cancellation preempt an amplifying operator at all.

Registering them in one place is the actual fix: a new construction site now has to opt out to be wrong, and SessionStateBuilder is consumed and returned so this composes into an existing chain.

Note on SpillableJoinSelection::from_capacity(): it reads the calling process’s cgroup. On an executor that is right; on the coordinator it describes the coordinator, not the executors the plan will run on. The threshold is overridable via KRISHIV_SPILL_JOIN_BUILD_BYTES, which is the supported way to make coordinator-side planning use the executor’s real per-task share until the capacity is plumbed through the stage builder.