Crate baseRT 

Safe Rust bindings for the BaseRT LLM inference engine.

BaseRT is a C++17 inference engine optimized for Apple Silicon via Metal compute shaders. This crate provides a safe, idiomatic Rust wrapper around the C API.

Example

use baseRT::{Model, SamplingConfig};

let model = Model::load("model.base", None, 0).unwrap();
let tokens = model.encode("Hello, world!").unwrap();

let stats = model.generate(&tokens, 256, SamplingConfig::greedy(), |_id, text| {
    print!("{text}");
    true
}).unwrap();

println!("\n({:.0} tok/s)", stats.decode_tokens_per_sec);

Structs

GenerationStats

Generation result statistics.

Model

An BaseRT model loaded on the GPU.

ModelConfig

Model configuration extracted from weight file metadata.

SamplingConfig

Sampling parameters for text generation.

TensorIter

Iterator over model tensors.

TranscribeSegment

One segment of the last transcription (verbose_json surface), with the text copied into owned memory.

TranscribeStats

Transcription result statistics.

Enums

Error

Error type for BaseRT operations.

Type Aliases

Result