Struct Llama32Runner 

pub struct Llama32Runner { /* private fields */ }

Implementations

impl Llama32Runner

pub fn builder() -> Llama32RunnerBuilder

pub fn config(&self) -> &Llama32Config

pub fn device(&self) -> Device

pub fn predict_logits(&mut self, prompt_ids: &[u32]) -> Result<Vec<f32>, Error>

Single prefill forward; returns last-position logits [vocab].

pub fn generate_packed( &mut self, prompt_ids: &[u32], n_new: usize, on_token: impl FnMut(u32), ) -> Result<Vec<u32>, Error>

pub fn generate( &mut self, prompt_ids: &[u32], n_new: usize, on_token: impl FnMut(u32), ) -> Result<Vec<u32>, Error>

Trait Implementations

impl LmRunner for Llama32Runner

fn family(&self) -> &'static str

Short family identifier matching rlx-cli::arch_runner_name (e.g. "qwen3", "qwen35", "gemma", "llama32"). Useful for logging / metrics / per-family branches in the caller.

fn vocab_size(&self) -> usize

LM head vocab size — useful for callers that need to size a logit buffer or validate token ids before calling Self::predict_logits. PLAN.md M9.

fn predict_logits(&mut self, prompt_ids: &[u32]) -> Result<Vec<f32>, Error>

Run prefill on prompt_ids and return the last-token logits over the full vocab. Mirrors the existing predict_logits method on every per-family runner.

fn generate( &mut self, prompt_ids: &[u32], n_new: usize, on_token: &mut dyn FnMut(u32) -> bool, ) -> Result<Vec<u32>, Error>

Generate up to n_new tokens after prompt_ids using greedy (argmax) sampling. on_token fires once per generated token and returns true to continue, false to stop. Returns the generated id sequence (excluding the prompt).

fn supports_multimodal(&self) -> bool

Whether this runner supports multimodal (image+text) generation via Self::generate_multimodal. Default false. Per-family runners that wire a vision encoder (e.g. Qwen35Runner with an mmproj path) override to true.

fn generate_multimodal( &mut self, _prompt: &str, _rgb: &[u8], _img_w: usize, _img_h: usize, _tokenizer: Option<&Path>, _n_new: usize, _on_token: &mut dyn FnMut(u32) -> bool, ) -> Result<Vec<u32>, Error>

Multimodal text generation: prefill the trunk with prompt text where image markers are spliced with vision embeddings derived from rgb (raw RGB bytes, row-major [h, w, 3]). Streams one token per on_token call; returns the full produced sequence.