Module cluster 

Cluster mode on Apache DataFusion Ballista (ADR-0013).

OxideLake does not write a scheduler. This module builds the pieces Ballista lets us plug in — a session builder that installs the placement rule for the declared cluster capability, a physical codec that ships Gpu*Exec nodes, and the executor function registry — and wires them into Ballista’s scheduler and executor processes, plus an in-process standalone cluster for tests and the CLI.

Functions

ballista_codec

Ballista’s combined logical + physical codec with oxide_codec as the physical half.

build_session_state

Builds the scheduler-side session state: Ballista’s defaults plus the placement rule targeting the declared cluster capability. This is where Gpu*Exec nodes enter a distributed plan.

config_producer

Ballista ConfigProducer yielding session_config.

executor_config

Executor configuration with OxideLake’s hooks installed.

function_registry

Functions registered on every executor: Ballista’s defaults plus OxideLake’s SQL UDFs (l2_distance, cosine_distance), so a distance projection that was not lowered to a Gpu*Exec still runs on workers.

oxide_codec

The physical codec every OxideLake process installs: our Gpu*Exec encoding on top of Ballista’s own codec (shuffle nodes).

run_executor

Runs a Ballista executor until it exits.

run_scheduler

Runs a Ballista scheduler until it exits.

runtime_producer

Ballista RuntimeProducer for executors.

scheduler_config

Scheduler configuration with OxideLake’s hooks installed.

session_builder

Ballista SessionBuilder wrapping build_session_state.

session_config

Session configuration for every process: Ballista defaults, Parquet pruning on, and the OxideLake codec.

start_standalone

Starts an in-process scheduler plus executors executors on ephemeral localhost ports and returns the scheduler address. Everything runs on the current tokio runtime and stops with it.