Trait StreamClient

Source

pub trait StreamClient: Send + Sync {
    // Required methods
    fn stream<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>(
        &'life0 self,
        messages: &'life1 [ChatMessage],
        tools: &'life2 [Value],
        reasoning: Option<&'life3 ReasoningConfig>,
        response_format: Option<&'life4 ResponseFormat>,
    ) -> Pin<Box<dyn Future<Output = AgentResult<Pin<Box<dyn Stream<Item = AgentResult<StreamChunk>> + Send>>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait,
             'life4: 'async_trait;
    fn capabilities(&self) -> LlmCapabilities;

    // Provided methods
    fn chat<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>(
        &'life0 self,
        messages: &'life1 [ChatMessage],
        tools: &'life2 [Value],
        reasoning: Option<&'life3 ReasoningConfig>,
        response_format: Option<&'life4 ResponseFormat>,
    ) -> Pin<Box<dyn Future<Output = AgentResult<String>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait,
             'life4: 'async_trait { ... }
    fn model_name(&self) -> &str { ... }
}

Expand description

Provider-agnostic streaming client trait.

This is the recommended interface for LLM provider integration. Providers only need to implement stream — chat has a default implementation that collects text deltas from the stream.

This follows the Rust standard library convention: Iterator only requires next(), std::io::Read only requires read().

The older LlmClient trait is still supported via LlmClientAdapter.

Required Methods§

Source

fn stream<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>( &'life0 self, messages: &'life1 [ChatMessage], tools: &'life2 [Value], reasoning: Option<&'life3 ReasoningConfig>, response_format: Option<&'life4 ResponseFormat>, ) -> Pin<Box<dyn Future<Output = AgentResult<Pin<Box<dyn Stream<Item = AgentResult<StreamChunk>> + Send>>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, 'life4: 'async_trait,

Stream LLM response chunks from the provider.

This is the only required method. Implementors translate their provider’s SSE/streaming protocol into StreamChunk events.

Source

fn capabilities(&self) -> LlmCapabilities

Return the provider’s capabilities.

Provided Methods§

Source

fn chat<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>( &'life0 self, messages: &'life1 [ChatMessage], tools: &'life2 [Value], reasoning: Option<&'life3 ReasoningConfig>, response_format: Option<&'life4 ResponseFormat>, ) -> Pin<Box<dyn Future<Output = AgentResult<String>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, 'life4: 'async_trait,

Convenience: collect all StreamChunk::Text deltas into a single string.

The default implementation streams and concatenates text chunks. Providers may override this with a dedicated non-streaming API call for better latency or cost.

Source