OxideLake

A GPU-accelerated, Arrow-native distributed analytical query engine, written in Rust. It reads and writes Parquet on object storage; a catalog, a metastore and transactions are out of scope, so it is a query engine over a lake rather than a lakehouse in the full sense.

Status: internal phases 0–7 complete; published 0.2.0. Production readiness is milestone v0.2.0, and 0.2.0 did not finish it — ten Phase 8 lines remain open, three of them needing a CUDA device to verify on. What 0.2.0 does add is the ability to tell what a deployment is actually doing: a CPU-fallback counter, placement notes under oxide explain, query spans, a per-query log line and a Prometheus endpoint. The default build is pure CPU and the whole gate is green on Linux x86_64 and macOS arm64. Both GPU backends have executed on real hardware: Metal on Apple silicon (conformance-tested against stock DataFusion on an M4 Pro) and CUDA on an NVIDIA T4 since 2026-09-06, where every kernel was NVRTC-compiled and launched — and where a correctness bug in the aggregation kernel was found. STATUS.md holds the verification matrix, component by component, and is the ledger ADR-0012 makes it. The specification the engine was built to is docs/SPEC.md.

Install

brew install vyncint/tap/oxidelake       # macOS and Linux: static binaries, no Rust toolchain
cargo install oxidelake-runtime          # or build from crates.io

Both give you the oxide, oxide-scheduler and oxide-worker binaries. The Homebrew formula is generated by the release itself from the archives' checksums, so it never describes a build that did not happen.

Or download a static archive for Linux (x86_64, aarch64, musl) or macOS (Apple silicon, Intel) from the latest release, verify it against the .sha256 beside it, and put the three binaries on your PATH. Using the engine as a library is cargo add oxidelake-api.

What OxideLake is

OxideLake runs the same plans in two modes over the Apache Arrow columnar memory model:

• Embedded mode (DuckDB-like) — in-process DataFusion; operators exchange Arc<RecordBatch> over bounded async channels with no serialization.
• Cluster mode (Spark-like) — Apache DataFusion Ballista schedules stages across executors and streams shuffle partitions over Arrow Flight; OxideLake supplies the GPU operators, the placement rule and the plan codec that ships them to executors.

Acceleration backends: CPU (always available, the correctness reference), NVIDIA CUDA (opt-in feature, NVRTC-JIT at runtime), Apple Metal on unified memory (opt-in, macOS only, MSL compiled at runtime).

The differentiator is the operator layer: GpuFilterExec (fused filter + projection), GpuHashJoinExec, GpuAggregateExec and GpuVectorDistanceExec are DataFusion ExecutionPlans with their own CUDA and Metal kernels behind one object-safe hardware abstraction. A placement rule rewrites eligible plan nodes for the target backend — EXPLAIN shows GpuFilterExec[metal] — and every operator still selects the real local backend at execution time and falls back per batch to the CPU reference, which is what keeps a heterogeneous cluster correct. Storage is Parquet pruned by DataFusion's statistics, page index and Bloom filters, with Arrow IPC for zero-decode spill and caches.

Quickstart

Everything below runs as written (Rust 1.98 via rust-toolchain.toml; the workspace builds on 1.94.1 and up — or just make quickstart):

cargo build --release -p oxidelake-runtime          # or `cargo install oxidelake-runtime`

# 1M-row demo table: Parquet with Bloom filters on id and k, page statistics on
./target/release/oxide gen-data --rows 1000000 --out data/

# embedded SQL over it
./target/release/oxide sql -q "SELECT k, SUM(v) FROM t GROUP BY k ORDER BY k" --table t=data/

# vector search: l2_distance / cosine_distance are built-in SQL UDFs
./target/release/oxide sql -q "SELECT id, l2_distance(emb, [1.0,0.0,2.0,0.5,-1.0,0.25,3.0,0.0]) AS d \
  FROM t ORDER BY d LIMIT 5" --table t=data/

# what the plan looks like for a GPU cluster: placement tags without needing the GPU
./target/release/oxide explain -q "SELECT k, SUM(v) FROM t WHERE k >= 2 AND v < 4.0 GROUP BY k" \
  --table t=data/ --target cuda

# the terminal dashboard over a real query: plan DAG, per-operator telemetry, column profiles
./target/release/oxide tui -q "SELECT k, SUM(v) FROM t WHERE k >= 2 GROUP BY k" --table t=data/ --target cuda

In-database inference (optional, --features predict)

Score a row against a trained model without leaving SQL. Off by default; it pulls in a tensor library.

cargo build --release -p oxidelake-runtime --features predict

# an oxmera Sequential/Linear model saved with safetensors, whose header
# declares {"__metadata__": {"oxidelake.activation": "relu"}}
./target/release/oxide sql --table t=data/ \
  -q "SELECT id, predict('scorer.safetensors', emb) AS logits FROM t LIMIT 5"

# a checkpoint whose header says nothing: state the activation in the query
./target/release/oxide sql --table t=data/ \
  -q "SELECT id, predict('scorer.safetensors', emb, 'gelu') AS logits FROM t LIMIT 5"

+----+--------------------------+
| id | logits                   |
+----+--------------------------+
| 0  | [-31.006792, 3.9050333]  |
| 1  | [7.2003703, -4.237343]   |
+----+--------------------------+

It composes with everything else — including the vector search that was already here, which is the point: OxideLake could find the nearest rows and could not score them.

./target/release/oxide sql --table t=data/ -q \
  "SELECT id, l2_distance(emb, [1.0,0.0,2.0,0.5,-1.0,0.25,3.0,0.0]) AS dist,
          predict('scorer.safetensors', emb)[2] AS score
   FROM t ORDER BY dist LIMIT 5"

The model file is read for its architecture, not just its weights: predict rebuilds an oxmera::nn::Sequential of Linear layers from the 0.weight, 1.weight, … naming, and takes the activation between them from the safetensors __metadata__ key oxidelake.activation — relu, gelu, sigmoid, tanh or none, applied between layers and never after the last. A file that declares nothing is refused, because the alternative is scoring it with an activation it was not trained with and returning a confident wrong number; the third argument predict(path, features, 'relu') is how you say it yourself, and it must agree with the file when the file has an opinion. The CPU-only execution, and why this is a UDF rather than an operator, are recorded in ADR-0015.

--target cpu|cuda|metal picks the placement target; execution always uses the hardware that is present (operators planned for an absent GPU take the per-batch CPU path — exactly what cluster executors do with a scheduler's plan).

Cluster mode

# OXIDE_CLUSTER_BACKEND declares the cluster's placement capability (default: cpu, no rewrites)
OXIDE_CLUSTER_BACKEND=cuda ./target/release/oxide-scheduler --port 50050 &
./target/release/oxide-worker --scheduler-port 50050 --port 50051 --grpc-port 50052 &
./target/release/oxide sql --cluster df://127.0.0.1:50050 \
  -q "SELECT k, SUM(v) FROM t GROUP BY k ORDER BY k" --table t=data/

The end-to-end test in crates/oxidelake-runtime/tests/cli.rs spawns exactly this topology on ephemeral ports and asserts the cluster output is byte-identical to embedded mode.

DataFrame API

use oxidelake_api::prelude::*;

let session = OxideSession::local()?;
session.register_parquet("t", "data/").await?;
let nearest = session
    .table("t").await?
    .filter(col("k").gt_eq(lit(2)))?
    .vector_distance("emb", &[0.0; 8], DistanceMetric::L2, "d")?
    .sort(vec![col("d").sort(true, false)])?
    .limit(10)?
    .collect().await?;

The fluent verbs build ordinary DataFusion plans, so the placement rule lowers them the same way it lowers SQL — .vector_distance(…) plans GpuVectorDistanceExec[metal] on a Metal target.

GPU backends

# CUDA: compiles and lints with no CUDA installed (cudarc dynamic loading + NVRTC)
cargo check -p oxidelake-runtime --features cuda

# Metal (macOS): the conformance suite executes on the device
cargo test -p oxidelake-compute --features metal -- --ignored

# On a CUDA machine (run on a T4 since 2026-09-06; see STATUS.md):
OXIDE_BACKEND=cuda cargo test -p oxidelake-compute --features cuda -- --ignored

Configuration

Every knob OxideLake reads, in one table. Flags win over environment variables; an explicit choice that the machine cannot honour is a startup error, never a silent fallback.

oxide

oxide-scheduler

oxide-worker

Environment

Repository map

Verification

Every change lands behind required-green and commit-policy. make gate runs formatting, feature-specific Clippy, the test suites, all-feature rustdoc, dependency-coherence and supply-chain checks, plus release and CI script tests. CI separates the Linux feature tests and checks the MSRV; macOS runs Metal and PTY tests and executes conformance when a Metal device exists. Dependency caching reuses compilation while tests still run; release CI retains a clean full gate. Known documentation-only changes may skip Rust jobs under the tested verification policy. The honesty rule applies throughout: nothing is claimed as working unless it ran; GPU paths that only compiled are recorded as exactly that in STATUS.md. The 2026-09-02 security and performance audit, with every finding's status, is in docs/audit-2026-09-02.md.

Contributing

See CONTRIBUTING.md for the workflow (make help lists every target), SECURITY.md for the trust model and how to report vulnerabilities, and CHANGELOG.md for what changed.

License

Apache-2.0 — see LICENSE.