BaseRT Rust Bindings

Rust bindings for the BaseRT LLM inference engine (Apple Silicon / Metal).

Two crates are provided:

• baseRT-sys -- raw #[repr(C)] FFI bindings (unsafe).
• baseRT -- safe, idiomatic Rust wrapper with RAII, Result error handling, and closures for streaming.

Prerequisites

Build the single shared engine so that libbaseRT.dylib exists:

cd /path/to/baseRT
make shared

By default the crates dynamically link the one libbaseRT.dylib, which already carries the Metal kernels (embedded metallib) and its own framework / C++ dependencies — one redistributable artifact, matching the Python / Node / Swift bindings. Point the build at the directory containing it with BASERT_LIB_DIR. When it is unset, the build uses the first of these that holds the library: ../../../build relative to the sys crate, a prebuilt engine from install.sh ($BASERT_INSTALL_DIR, default ~/.basert), or Homebrew ($(brew --prefix)/opt/basert/libexec). So after brew install basecompute/tap/basert, cargo add baseRT builds with no extra setup.

export BASERT_LIB_DIR=/path/to/baseRT/build

The baseRT crate embeds an rpath to that directory, so its own examples and tests run without DYLD_LIBRARY_PATH. A downstream binary that depends on baseRT must make the dylib discoverable at runtime — either add an rpath in its own build.rs (cargo:rustc-link-arg=-Wl,-rpath,<dir>), install the dylib to a standard location, or set DYLD_LIBRARY_PATH=$BASERT_LIB_DIR.

To link the static engine archive instead (fully static, pulls in the frameworks manually), enable the static feature:

baseRT = { path = "...", features = ["static"] }

A crate that loads libbaseRT itself at run time can depend on baseRT-sys with the no-link feature: the build then looks for no library and links nothing. The types stay usable; the extern functions must not be called.

Either way, baseRT-sys tells dependents' build scripts where the engine's C headers are: DEP_BASERT_INCLUDE holds baseRT/baseRT.h.

Usage

Add the dependency from crates.io (install the engine first, e.g. brew install basecompute/tap/basert):

[dependencies]
baseRT = "0.4"

or from a checkout:

[dependencies]
baseRT = { path = "path/to/baseRT/bindings/rust/baseRT" }

Run the bundled example end-to-end:

BASERT_LIB_DIR=/path/to/baseRT/build \
  cargo run --example smoke -- /path/to/model.base

Text generation

use baseRT::{Model, SamplingConfig};

fn main() -> baseRT::Result<()> {
    // metallib path = None → use the metallib embedded in libbaseRT.dylib.
    let model = Model::load("models/Qwen3-0.6B-Q4_0.base", None, 0)?;

    println!("Architecture: {}", model.architecture());
    println!("GPU memory:   {:.1} MB", model.memory() as f64 / 1e6);

    let tokens = model.encode("The meaning of life is")?;

    let stats = model.generate(&tokens, 128, SamplingConfig::greedy(), |_id, text| {
        print!("{text}");
        true // return false to stop early
    })?;

    println!("\n\nPrefill: {:.0} tok/s", stats.prefill_tokens_per_sec);
    println!("Decode:  {:.0} tok/s", stats.decode_tokens_per_sec);
    Ok(())
}

Non-streaming generation

let (text, stats) = model.generate_text(&tokens, 128, SamplingConfig::greedy())?;
println!("{text}");

Multi-turn chat (continue from KV cache)

let turn1 = model.encode("User: Hi!\nAssistant:")?;
model.generate(&turn1, 64, SamplingConfig::greedy(), |_, t| { print!("{t}"); true })?;

let turn2 = model.encode("\nUser: Tell me more.\nAssistant:")?;
model.generate_continue(&turn2, 64, SamplingConfig::greedy(), |_, t| { print!("{t}"); true })?;

Whisper transcription

let model = Model::load("models/whisper-base-en.bin", None, 0)?;
assert!(model.is_whisper());

let (text, stats) = model.transcribe("audio.wav", Some("en"))?;
println!("{text}");
println!("Decode: {:.1} ms", stats.decode_ms);

Model inspection

println!("Tensors: {}", model.tensor_count());
for (name, dtype) in model.tensors() {
    println!("  {name}: dtype={dtype}");
}

Sampling configuration

// Greedy (default)
let cfg = SamplingConfig::greedy();

// Temperature sampling
let cfg = SamplingConfig::with_temperature(0.7);

// Full control
let cfg = SamplingConfig {
    temperature: 0.8,
    top_k: 50,
    top_p: 0.95,
    min_p: 0.05,
    repeat_penalty: 1.1,
    presence_penalty: 0.0,
    frequency_penalty: 0.0,
    seed: 0,                 // 0 = wall-clock-seeded; non-zero = deterministic
    logit_bias: Vec::new(),  // e.g. vec![(token_id, +5.0), (banned_id, -100.0)]
};

Low-level API

let first_token = model.prefill(&tokens);
let next = model.decode_step(first_token, tokens.len() as i32);

// Or batch decode
let batch = model.chain_decode(first_token, tokens.len() as i32, 16)?;

Error handling

All fallible operations return baseRT::Result<T>. Errors come from:

• Error::Api(msg) -- the C API returned NULL or an error (message from baseRT_get_error()).
• Error::InvalidString(e) -- a Rust string contained an interior NUL byte.