Trait ChatModel 

pub trait ChatModel: Send + Sync {
    // Required method
    fn complete_streaming<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: &'life1 CompletionRequest,
        delta_tx: UnboundedSender<(String, bool)>,
    ) -> Pin<Box<dyn Future<Output = Result<CompletionResponse, LlmError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
}

Anything that can turn a chat-completion request into a response.

Required Methods

fn complete_streaming<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 CompletionRequest, delta_tx: UnboundedSender<(String, bool)>, ) -> Pin<Box<dyn Future<Output = Result<CompletionResponse, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stream cumulative (content, has_tool_calls) updates and return the canonical terminal response. This is the only model invocation path.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementations on Foreign Types

impl ChatModel for Arc<dyn ChatModel>

A type-erased model, so callers (e.g. an HTTP service holding one Agent in shared state) can pick the backend at runtime — real client vs stub vs replay — without the loop being generic over every concrete type.

fn complete_streaming<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 CompletionRequest, delta_tx: UnboundedSender<(String, bool)>, ) -> Pin<Box<dyn Future<Output = Result<CompletionResponse, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

impl ChatModel for LlmClient

The production model is the real LLM client.

fn complete_streaming<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 CompletionRequest, delta_tx: UnboundedSender<(String, bool)>, ) -> Pin<Box<dyn Future<Output = Result<CompletionResponse, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Implementors