Struct QwenChatProvider 

pub struct QwenChatProvider { /* private fields */ }

A real local chat backend: a Qwen2 GGUF model + its tokenizer, driven through el_runtime::InferenceSession.

Each chat call renders the whole conversation to Qwen2.5 ChatML, builds a fresh QwenEngine (candle has no public KV-cache reset), then runs the SDK’s standard provenance-gated session: load_prompt (prefill) → generate (grammar mask → safety steer → guard + checkpointed rollback → greedy commit). On-device safety (ADR-005 Lightweight tier + the ADR-012 control loop) is on by default; see with_safety. The provider holds no mutable session state, so it is Send + Sync without locking.

Implementations

impl QwenChatProvider

pub fn from_paths( model_path: impl AsRef<Path>, tokenizer_path: impl AsRef<Path>, ) -> Result<Self>

Load a Qwen2 GGUF model and its tokenizer.json from local paths.

pub fn with_safety(self, mode: SafetyMode) -> Self

Select the on-device safety tier (ADR-005). SafetyMode::Off disables the steerer and the ADR-012 control loop (the plain single-pass decode); SafetyMode::Lightweight (the default) runs the token-anchor guard + hard-ban steerer + checkpointed rollback. SecDecoding/Csd need model assets not shipped here and fall back to the Lightweight wiring.

pub fn with_extra_guard_words<I, S>(self, words: I) -> Self

where I: IntoIterator<Item = S>, S: AsRef<str>,

Add extra words to the chunk guard’s unsafe patterns (resolved to token ids via this model’s tokenizer). Primarily a test/demo hook: e.g. --guard-word banana lets you watch the ADR-012 rollback / fail-closed refusal fire on a benign word, without needing the model to emit genuinely harmful content. Guard-only — these are not added to the hard-ban list, so the trajectory loop (not silent suppression) is what engages.

pub fn with_expert_model(self, path: impl AsRef<Path>) -> Self

Enable model-backed contrastive steering (ADR-013) with a safety expert GGUF (same tokenizer/family as the chat model). Steering runs only inside the early-token window. Pointing this at the chat model itself gives ~zero contrast (a no-op); a safety-tuned Qwen GGUF gives real steering. No effect under --safety off.

pub fn with_steer_alpha(self, alpha_milli: i32) -> Self

Contrastive steering strength ×1000 (1000 = 1.0×). Only meaningful with with_expert_model.

Trait Implementations

impl LlmProvider for QwenChatProvider

fn chat(&self, req: &ChatRequest) -> Result<ChatResponse>

Blocking, non-streaming chat completion.

fn chat_stream( &self, req: &ChatRequest, on_token: &mut dyn FnMut(ChatToken), ) -> Result<()>

Streaming chat: calls on_token for each fragment as it arrives. Returns when generation is complete or on error. The final call will have ChatToken::is_final == true.