rlx-onnx

Run ONNX (.onnx) models on RLX backends. Default path: import via rlx-onnx-import, compile with rlx-runtime Session, execute on CPU / Metal / CUDA / ROCm / MLX / wgpu. Optional ONNX Runtime (ort) backend for parity and execution-provider fallback.

Install

[dependencies]
rlx-onnx = { version = "0.2", default-features = true, features = ["cpu"] }

# GPU backends (pick what your host supports)
# rlx-onnx = { version = "0.2", features = ["cuda"] }
# rlx-onnx = { version = "0.2", features = ["metal", "mlx"] }

# ORT fallback + EPs
# rlx-onnx = { version = "0.2", features = ["ort-fallback", "ort-cuda"] }

From the rlx prelude:

rlx = { version = "0.2", features = ["cpu", "onnx"] }

Usage

use rlx_onnx::{OnnxCompileLevel, OnnxModel};
use rlx_runtime::Device;

let mut model = OnnxModel::load_native("model.onnx", Device::Cpu, OnnxCompileLevel::Level3, 128)?;
model.print_io();
let inputs = model.zero_inputs_sized(32)?;
let outputs = model.run(&inputs)?;

• OnnxCompileLevel — maps ONNX graph-opt tiers 0–3 to CompileOptions (DCE / constant folding / full pipeline).
• OnnxExecBackend::Native (default) — RLX compile + execute.
• OnnxExecBackend::Ort — requires ort or ort-fallback feature.

CLI

cargo run -p rlx-onnx --features native --bin rlx-onnx-run -- model.onnx \
  --device cpu --level 3 --seq-len 128 --warmup 1 --iters 5

# List inputs/outputs only
cargo run -p rlx-onnx --bin rlx-onnx-run -- model.onnx --list-io

# ORT path (rebuild with ort-fallback)
cargo run -p rlx-onnx --features ort-fallback --bin rlx-onnx-run -- \
  model.onnx --exec ort --device cuda

Features

Tests

cargo test -p rlx-onnx

License

GPL-3.0-only.