Struct QwenEngine 

pub struct QwenEngine { /* private fields */ }

A real Qwen2 transformer InferenceEngine.

Holds candle’s stateful KV cache. Within one generation it is fed incrementally (prefill, then one new token per next_logits call); candle exposes no public cache reset, so a fresh conversation builds a new engine. Float logits are quantised to integer milli-logits at the seam, exactly like CandleEngine, so the runtime stays float-free.

Implementations

impl QwenEngine

pub fn from_path(path: impl AsRef<Path>, eos: Token) -> Result<Self>

Load Qwen2 weights from a consumer-supplied GGUF file.

Trait Implementations

impl InferenceEngine for QwenEngine

fn prefill(&mut self, tokens: &[Token]) -> Result<u32>

Encode the (compressed) prompt; returns the resulting KV length.

fn next_logits(&mut self, committed: &[Token]) -> Vec<i32>

Produce next-token logits given the committed context.

fn eos_token(&self) -> Token

The end-of-sequence token id.