rlx-pkg

RLX ship format (.rlxp): flat mmap by default with hybrid hot / warm / cold storage, optional ZIP / directory layouts, and an optional executable MIR graph (ONNX-style graph + weights, or weight-only).

Spec: docs/rlxp.md.

Warm compression is for rarely touched host blobs — not a substitute for GGUF-class quants (Q4_K, …). Keep active LLM weights hot + packed.

Usage

use rlx_pkg::{
    Package, PackedWeight, StorageTier, WriteOptions, write_package,
};

let mut opts = WriteOptions {
    name: "model".into(),
    // include_graph: true by default — set false for weight-only packs
    ..WriteOptions::default()
};
opts.sidecars.push(("tokenizer".into(), "application/json".into(), tok));

write_package(
    "model.rlxp",
    &graph,
    &[
        PackedWeight::hot("w", shape, "f32", "row_major", bytes),
        PackedWeight {
            name: "expert".into(),
            tier: StorageTier::Warm,
            ..PackedWeight::hot("expert", eshape, "f32", "row_major", ebytes)
        },
    ],
    &opts,
)?;

let pack = Package::open("model.rlxp")?;
pack.advise_hot_willneed();                 // optional page prefetch
let _ = pack.tensor_mmap("w")?;             // zero-copy hot
let _ = pack.tensor_bytes("expert")?;       // inflate warm (parallel for large)
let g = pack.graph()?;                      // materializes hot Constants by default

Runtime compile:

use rlx_runtime::{Device, Session};
use rlx_runtime::pkg::compile_rlxp;

let session = Session::new(Device::Cpu);
let compiled = compile_rlxp(&session, "model.rlxp")?;

CLI

cargo run -p rlx-pkg -- inspect model.rlxp
cargo run -p rlx-pkg -- verify model.rlxp
cargo run -p rlx-pkg -- import-gguf model.gguf -o model.rlxp --no-graph
cargo run -p rlx-pkg -- convert in.zip -o out.rlxp --container flat
cargo run -p rlx-pkg -- tier model.rlxp -o out.rlxp --warm expert --hot tok_embd

ONNX → RLXP (optional executable graph) is on rlx-bake:

cargo run -p rlx-bake --features onnx -- import-onnx model.onnx -o model.rlxp
cargo run -p rlx-bake --features onnx -- import-onnx model.onnx -o w.rlxp --no-graph

Features

Benches / examples

just throttle
RLX_ALLOW_THROTTLE=1 cargo run -p rlx-pkg --example bench_hybrid --release
RLX_ALLOW_THROTTLE=1 cargo run -p rlx-pkg --example bench_compare --release
RLX_ALLOW_THROTTLE=1 cargo bench -p rlx-pkg --bench hybrid_load --bench q4k_compare
# MNIST end-to-end (bake + RLXP load/compile/run):
RLX_ALLOW_THROTTLE=1 cargo run -p rlx-bake --example mnist_bench_rlxp --features runtime --release

Python (pyrlx)

import pyrlx
pyrlx.load_rlxp("model.rlxp")
pyrlx.convert_gguf_to_rlxp("m.gguf", "m.rlxp", include_graph=False)

License

MIT OR Apache-2.0.