Struct CandleEngine 

pub struct CandleEngine { /* private fields */ }

Candle-backed inference engine.

Implementations

impl CandleEngine

pub fn toy(vocab: usize, dim: usize, eos: Token) -> Result<Self>

Build a deterministic toy model on the CPU — no model file required.

Uses fixed synthetic weights so tests are deterministic.

pub fn from_path(path: impl AsRef<Path>, eos: Token) -> Result<Self>

Load token_embd.weight and output.weight from a consumer-supplied GGUF file.

Limitations

This engine’s forward pass is embed[last_token] · w_out — a single linear projection. Only these two tensors are used; transformer blocks, attention, RoPE, and norms present in the GGUF are ignored. Logits will not match a real Llama/Mistral/etc. forward. This is the ADR-002 engine-seam proof; for a full transformer forward implement a separate InferenceEngine using candle-transformers.

pub fn from_bytes(data: &[u8], eos: Token) -> Result<Self>

Load from raw bytes (WASM / memory-mapped scenarios).

Same limitations as Self::from_path: only token_embd.weight and output.weight are used; the forward is embed[last] · w_out.

Trait Implementations

impl InferenceEngine for CandleEngine

fn prefill(&mut self, tokens: &[Token]) -> Result<u32>

Encode the (compressed) prompt; returns the resulting KV length.

fn next_logits(&mut self, committed: &[Token]) -> Vec<i32>

Produce next-token logits given the committed context.

fn eos_token(&self) -> Token

The end-of-sequence token id.