Trait ChatDriver 

pub trait ChatDriver: Send + Sync {
    // Required method
    fn chat_completion_stream<'life0, 'life1, 'async_trait>(
        &'life0 self,
        messages: Vec<LlmMessage>,
        config: &'life1 LlmCallConfig,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<LlmStreamEvent, AgentLoopError>> + Send>>, AgentLoopError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             'life1: 'async_trait,
             Self: 'async_trait;

    // Provided methods
    fn chat_completion<'life0, 'life1, 'async_trait>(
        &'life0 self,
        messages: Vec<LlmMessage>,
        config: &'life1 LlmCallConfig,
    ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, AgentLoopError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             'life1: 'async_trait,
             Self: 'async_trait { ... }
    fn list_models<'life0, 'async_trait>(
        &'life0 self,
    ) -> Pin<Box<dyn Future<Output = Result<Option<Vec<DiscoveredModel>>, AgentLoopError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             Self: 'async_trait { ... }
    fn supports_compact(&self) -> bool { ... }
    fn supports_stateful_responses(&self) -> bool { ... }
    fn effective_context_window(&self, _model: &str) -> Option<usize> { ... }
    fn supports_parallel_tool_calls(&self, _model: &str) -> bool { ... }
    fn compact<'life0, 'async_trait>(
        &'life0 self,
        _request: CompactRequest,
    ) -> Pin<Box<dyn Future<Output = Result<Option<CompactResponse>, AgentLoopError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             Self: 'async_trait { ... }
}

Trait for LLM drivers

Implementations handle provider-specific API calls and response parsing.

Error contract

Drivers surface provider failures as AgentLoopError and classify them semantically at the provider boundary, where HTTP status and response body are still available:

• request-too-large conditions => AgentLoopError::request_too_large
• missing/unknown model => AgentLoopError::model_not_available
• everything else => AgentLoopError::llm_kind(LlmErrorKind::..., msg), using LlmErrorKind::from_provider_status (HTTP drivers) or LlmErrorKind::from_error_text (SDK drivers without a status). Plain AgentLoopError::llm is reserved for unclassifiable errors; downstream then falls back to string classification.

Quota/billing exhaustion (LlmErrorKind::QuotaExhausted) is non-transient and must not be retried by driver retry loops even when the provider reports it under a transient status like 429.

Required Methods

fn chat_completion_stream<'life0, 'life1, 'async_trait>( &'life0 self, messages: Vec<LlmMessage>, config: &'life1 LlmCallConfig, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<LlmStreamEvent, AgentLoopError>> + Send>>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Call the LLM with streaming response

Provided Methods

fn chat_completion<'life0, 'life1, 'async_trait>( &'life0 self, messages: Vec<LlmMessage>, config: &'life1 LlmCallConfig, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Call the LLM without streaming (convenience method)

fn list_models<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Option<Vec<DiscoveredModel>>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

List available models from the provider

Returns Ok(Some(models)) if the provider supports model listing, or Ok(None) if not supported (e.g., custom endpoints, proxies).

Implementations should filter to chat/completion models only, excluding embedding models, TTS, whisper, etc.

fn supports_compact(&self) -> bool

Check if this driver supports the compact endpoint

The compact endpoint compresses conversation history by replacing assistant messages, tool calls, and tool results with an encrypted compaction item. User messages are kept verbatim.

Returns true if the driver supports compaction, false otherwise. Currently only supported by OpenAI’s Responses API.

fn supports_stateful_responses(&self) -> bool

Whether this driver persists Responses API state and can resolve tool calls that are reachable only through previous_response_id.

Stateless and custom drivers default to false; they must receive a self-contained tool call/result transcript on every request.

fn effective_context_window(&self, _model: &str) -> Option<usize>

Effective context window for model, when the driver has authoritative model metadata that is not represented by Everruns’ built-in profiles.

External drivers should override this so host policy does not guess from a provider/model table that cannot describe their runtime model aliases.

fn supports_parallel_tool_calls(&self, _model: &str) -> bool

Whether this driver can express the request-level parallel_tool_calls preference on the wire for model.

Drivers that map the preference onto a request field (OpenAI/Anthropic families) return true; drivers whose provider API has no such control (Gemini, Bedrock) return false. When false, the preference is omitted from the request and is honored only by the local tool scheduler, so an avoid preference still serializes tool execution on every provider.

The default is false (conservative: omit unless a driver opts in).

fn compact<'life0, 'async_trait>( &'life0 self, _request: CompactRequest, ) -> Pin<Box<dyn Future<Output = Result<Option<CompactResponse>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Compact a conversation to reduce context size

This method compresses conversation history by calling the provider’s compact endpoint. User messages are kept verbatim, while assistant messages, tool calls, and tool results are replaced by an encrypted compaction item that preserves latent context but is opaque.

Arguments

• request - The compact request containing the model and input items

Returns

Returns Ok(Some(response)) if compaction succeeded, Ok(None) if compaction is not supported by this driver, or Err if an error occurred.

The response contains the compacted output items which can be used directly as input for the next chat completion call.

Trait Implementations

impl ChatDriver for Box<dyn ChatDriver>

Implement ChatDriver for Box<dyn ChatDriver> to allow dynamic dispatch

fn chat_completion_stream<'life0, 'life1, 'async_trait>( &'life0 self, messages: Vec<LlmMessage>, config: &'life1 LlmCallConfig, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<LlmStreamEvent, AgentLoopError>> + Send>>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

Call the LLM with streaming response

fn chat_completion<'life0, 'life1, 'async_trait>( &'life0 self, messages: Vec<LlmMessage>, config: &'life1 LlmCallConfig, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

Call the LLM without streaming (convenience method)

fn list_models<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Option<Vec<DiscoveredModel>>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

List available models from the provider

fn supports_compact(&self) -> bool

Check if this driver supports the compact endpoint

fn supports_stateful_responses(&self) -> bool

Whether this driver persists Responses API state and can resolve tool calls that are reachable only through previous_response_id.

fn effective_context_window(&self, model: &str) -> Option<usize>

Effective context window for model, when the driver has authoritative model metadata that is not represented by Everruns’ built-in profiles.

fn supports_parallel_tool_calls(&self, model: &str) -> bool

Whether this driver can express the request-level parallel_tool_calls preference on the wire for model.

fn compact<'life0, 'async_trait>( &'life0 self, request: CompactRequest, ) -> Pin<Box<dyn Future<Output = Result<Option<CompactResponse>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

Compact a conversation to reduce context size

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementations on Foreign Types

impl ChatDriver for Box<dyn ChatDriver>

Implement ChatDriver for Box<dyn ChatDriver> to allow dynamic dispatch

fn chat_completion_stream<'life0, 'life1, 'async_trait>( &'life0 self, messages: Vec<LlmMessage>, config: &'life1 LlmCallConfig, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<LlmStreamEvent, AgentLoopError>> + Send>>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

fn chat_completion<'life0, 'life1, 'async_trait>( &'life0 self, messages: Vec<LlmMessage>, config: &'life1 LlmCallConfig, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

fn list_models<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Option<Vec<DiscoveredModel>>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

fn supports_compact(&self) -> bool

fn supports_stateful_responses(&self) -> bool

fn effective_context_window(&self, model: &str) -> Option<usize>

fn supports_parallel_tool_calls(&self, model: &str) -> bool

fn compact<'life0, 'async_trait>( &'life0 self, request: CompactRequest, ) -> Pin<Box<dyn Future<Output = Result<Option<CompactResponse>, AgentLoopError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Box<dyn ChatDriver>: 'async_trait,

Implementors

impl ChatDriver for LlmSimDriver

impl ChatDriver for MockProvider

impl ChatDriver for OpenAIProtocolChatDriver

impl ChatDriver for OpenResponsesProtocolChatDriver