Struct LocalAdapter 

pub struct LocalAdapter { /* private fields */ }

A crate::ModelAdapter that runs a GGUF Qwen model entirely on this machine’s CPU via kopitiam-runtime — no network, no cloud account, no API key. This is what makes CLAUDE.md’s Offline First pipeline (“existing knowledge, then native Rust, then local AI, then cloud AI as the final fallback”) a real, callable rung rather than a promise: before this type existed, “local AI” had nothing behind it but crate::EchoAdapter, which invokes no model at all.

Construct via LocalAdapter::load; see this module’s docs for why depending on kopitiam-runtime here does not violate the Semantic Runtime’s dependency rule.

Implementations

impl LocalAdapter

pub fn gpu_output_head_active(&self) -> bool

Whether the output projection is running on the GPU for this model.

Exposed so the CLI can tell the user which path it actually took. The notice used to say “on CPU” unconditionally, which stopped being true the moment the output head could be offloaded — and a status line that is confidently wrong is worse than no status line.

pub fn load(path: impl AsRef<Path>) -> Result<Self>

Loads a GGUF Qwen model and its embedded tokenizer from path, once, and holds both for the lifetime of the returned adapter.

Errors

Returns Err — never panics — if path does not exist, is not a valid GGUF/SafeTensors file, is missing metadata kopitiam_runtime::QwenConfig::from_metadata requires, or has no embedded tokenizer.ggml.tokens vocabulary (tokenizer_from_gguf’s own error cases). This is deliberate, not incidental: per crate::ModelAdapter::complete’s own docs, a failing adapter is how kopitiam-workflow falls back to the next rung of the Offline First pipeline, so every failure mode here has to surface as Err rather than a panic that would take the whole workflow process down with it.

Trait Implementations

impl ModelAdapter for LocalAdapter

fn stream(&self, request: &CompletionRequest) -> Receiver<StreamChunk>

Streams the model’s reply token by token off a background thread, so a chat UI renders each token as kopitiam-runtime decodes it instead of freezing for the whole generation — the non-negotiable shape on a phone doing a few tokens per second (temp_ai_design.md §10.4).

How it stays off the foreground thread

The heavy state — model weights and tokenizer — lives behind Arcs (see LocalAdapter::model), so this clones the two Arcs (cheap: two refcount bumps, no weight copy) and moves them into a spawned std thread. That thread owns the channel’s Sender and runs [generate], whose on_token callback fires once per decoded token; each fire sends a StreamChunk::Token. On a clean finish it sends StreamChunk::Done; if generate errors it sends StreamChunk::Error instead. This is the AID-0028 actor discipline (worker owns the resource, streams over a channel, never blocks the caller) — no async runtime, just a thread and an mpsc.

Moving the Arcs (rather than borrowing self) is what lets the thread outlive this call safely: it holds its own strong references, so the weights stay alive for exactly as long as generation needs them even if the LocalAdapter itself is dropped meanwhile.

If the caller stops listening

generate’s on_token can’t itself abort the loop, so if the receiver is dropped mid-reply the send starts failing but generation runs to its max_new_tokens/EOS on the background thread before the thread exits. Wasted compute, never a hang or a panic — acceptable for phase 1; a cancellation token is a later refinement.

fn name(&self) -> &str

Stable identifier for this adapter (e.g. "local-qwen", "claude"), used in logs and recorded provenance — not necessarily the same as CompletionResponse::model, which names the specific model that answered.

fn complete(&self, request: &CompletionRequest) -> Result<CompletionResponse>

Sends request to the backend and returns its reply.