Trait LanguageModel 

pub trait LanguageModel: MaybeSendSync {
    // Required methods
    fn generate<'life0, 'life1, 'life2, 'life3, 'async_trait>(
        &'life0 self,
        conversation: &'life1 Conversation,
        params: &'life2 GenParams,
        tools: &'life3 [ToolDefinition],
    ) -> Pin<Box<dyn Future<Output = Result<ModelStream, LmError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait;
    fn cancel(&self);
    fn save_state<'life0, 'async_trait>(
        &'life0 self,
    ) -> Pin<Box<dyn Future<Output = Result<Vec<u8>, LmError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
    fn load_state<'life0, 'life1, 'async_trait>(
        &'life0 self,
        blob: &'life1 [u8],
    ) -> Pin<Box<dyn Future<Output = Result<(), LmError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
}

The swappable language-model capability: a chat context in, structured generation out incrementally.

This is the durable interface. A native engine (e.g. a llama.cpp context) and a hosted engine (a Rig agent, a browser Worker) all implement this one trait, so LmStage — and the pipeline above it — never names a concrete model.

Engines are worker-handles

An implementor is expected to be a thin handle to a long-lived worker that owns the model’s mutable decode state: a dedicated thread on native (a llama.cpp context is !Send, so the worker pattern is mandatory there), a Web Worker on wasm32. That is why generate takes &self — it hands the context to the worker and returns its delta stream — and why the worker outlives any single call.

Streaming is a barge-in requirement

generate yields a ModelDelta at a time rather than one buffer at the end: every item of the returned ModelStream is a preemption point the run loop can drop an in-flight generation at, so a user barging in stops the reply within one delta instead of after the whole turn. Dropping the stream is how the stage stops pulling; cancel is how the engine stops producing.

cancel is a control call (see Processor’s control-call carve-out): it maps to the engine’s abort callback / an atomic the decode loop checks, so it is synchronous, non-blocking, and safe to invoke directly from a stage’s decide_* where the barge-in is decided.

?Send on wasm32 matches pipecrab’s single-threaded execution model, so one implementation runs unchanged on a current-thread executor and in the browser, where Send bounds cannot be satisfied.

Required Methods

fn generate<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, conversation: &'life1 Conversation, params: &'life2 GenParams, tools: &'life3 [ToolDefinition], ) -> Pin<Box<dyn Future<Output = Result<ModelStream, LmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Generate a reply to conversation under params with tools available, yielding ModelDeltas.

tools are the tools configured on the stage. An implementation that wraps a higher-level agent (e.g. a Rig agent) keeps its own registered tools internal and uses them here directly, alongside any in tools.

Takes &self: like every Stage::perform, generation must not mutate observable state, so the run loop can drop an in-flight call — at any stream item — on a barge-in interrupt without tearing anything. Every item of the returned ModelStream is such a preemption point.

fn cancel(&self)

Control call: abort in-flight generation. Sync, non-blocking, idempotent.

Maps to the engine’s abort callback / an atomic the decode loop checks; the next generate starts clean. Safe to call from a stage’s synchronous decide_* — see the trait-level note.

fn save_state<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<u8>, LmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Checkpoint the worker’s session state (KV cache and any decode state) to an opaque, serialized blob.

fn load_state<'life0, 'life1, 'async_trait>( &'life0 self, blob: &'life1 [u8], ) -> Pin<Box<dyn Future<Output = Result<(), LmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Restore worker session state previously produced by save_state.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors