Struct CandleEngine 

pub struct CandleEngine { /* private fields */ }

Candle-backed inference engine.

Implementations

impl CandleEngine

pub fn toy(vocab: usize, dim: usize, eos: Token) -> Result<Self>

Build a deterministic toy model on the CPU — no model file required.

Uses fixed synthetic weights so tests are deterministic.

pub fn from_path(path: impl AsRef<Path>, eos: Token) -> Result<Self>

Load token_embd.weight and output.weight from a consumer-supplied GGUF file.

Limitations

This engine’s forward pass is embed[last_token] · w_out — a single linear projection. Only these two tensors are used; transformer blocks, attention, RoPE, and norms present in the GGUF are ignored. Logits will not match a real Llama/Mistral/etc. forward. This is the ADR-002 engine-seam proof; for a full transformer forward implement a separate InferenceEngine using candle-transformers.

pub fn from_bytes(data: &[u8], eos: Token) -> Result<Self>

Load from raw bytes (WASM / memory-mapped scenarios).

Same limitations as Self::from_path: only token_embd.weight and output.weight are used; the forward is embed[last] · w_out.

Trait Implementations

impl InferenceEngine for CandleEngine

fn rollback(&mut self, _keep_committed: u32) -> Result<()>

Stateless: this engine’s forward is embed[committed.last()] · w_out, so it holds no KV cache to restore — a rollback is a no-op.

fn reset_cache(&mut self) -> Result<()>

Stateless: no conversation cache to discard between turns (ADR-018).

fn prefill(&mut self, tokens: &[Token]) -> Result<u32>

Encode the (compressed) prompt; returns the resulting KV length.

fn next_logits(&mut self, committed: &[Token]) -> Vec<i32>

Produce next-token logits given the committed context.

fn eos_token(&self) -> Token

The end-of-sequence token id.

fn prefill_reuse(&mut self, full_context: &[u32]) -> Result<u32, EdgeError>

Prefill full_context, reusing the KV already cached for its longest matching prefix and feeding only the divergent suffix — cross-turn incremental prefill (ADR-018 AC-3). Returns the resulting KV length.