Trait ApiClient 

pub trait ApiClient: Send + Sync {
    // Required methods
    fn model(&self) -> String;
    fn stream_messages(
        &self,
        messages: Vec<Message>,
        system: Option<String>,
        tools: Option<Vec<ToolSchema>>,
    ) -> Pin<Box<dyn Stream<Item = Result<StreamEvent, ApiError>> + Send + 'static>>;
    fn create_message(
        &self,
        messages: Vec<Message>,
        system: Option<String>,
        tools: Option<Vec<ToolSchema>>,
    ) -> Pin<Box<dyn Future<Output = Result<Value, ApiError>> + Send + '_>>;

    // Provided method
    fn set_model(&self, _model: &str) -> bool { ... }
}

Interface for API clients that communicate with LLM providers.

Defines the contract for both streaming and non-streaming message requests. Implementations handle provider-specific details such as authentication headers, request body formatting, SSE event parsing, and error code mapping.

The trait is object-safe, so clients can be used as BoxedApiClient (Box<dyn ApiClient>) or SharedApiClient (Arc<dyn ApiClient>) without issues.

Streaming Contract

stream_messages returns a 'static Stream of StreamEvents. Concrete implementations must clone all necessary data from &self into the returned future — no borrows of &self are captured. This allows the stream to outlive the client reference.

Implementors

• Downstream crates provide implementations for concrete LLM providers.
• Mock implementations for testing can be found in test utilities.

Example

struct MyProviderClient {
    api_key: String,
    model: String,
}

impl ApiClient for MyProviderClient {
    fn model(&self) -> String {
        self.model.clone()
    }

    fn stream_messages(
        &self,
        messages: Vec<Message>,
        system: Option<String>,
        tools: Option<Vec<ToolSchema>>,
    ) -> Pin<Box<dyn Stream<Item = Result<StreamEvent, ApiError>> + Send + 'static>> {
        // Clone data from &self, then build and return a stream
        let model = self.model.clone();
        let api_key = self.api_key.clone();
        // ... provider-specific streaming logic
        todo!()
    }

    fn create_message(
        &self,
        messages: Vec<Message>,
        system: Option<String>,
        tools: Option<Vec<ToolSchema>>,
    ) -> Pin<Box<dyn Future<Output = Result<serde_json::Value, ApiError>> + Send + '_>> {
        // Non-streaming fallback
        todo!()
    }
}

See Also

• BoxedApiClient — owned, single-threaded type alias.
• SharedApiClient — shared, multi-threaded type alias.
• ApiError — error type for API failures.
• StreamEvent — streaming event types returned by ApiClient::stream_messages.

Required Methods

fn model(&self) -> String

Get the model identifier for this client.

Returns the provider-specific model string (e.g., "llm-1", "llm-2", "llm-3"). Used by the framework for logging, token estimation, and fallback routing via FallbackManager.

Called by the framework during initialization and on each turn for observability purposes.

fn stream_messages( &self, messages: Vec<Message>, system: Option<String>, tools: Option<Vec<ToolSchema>>, ) -> Pin<Box<dyn Stream<Item = Result<StreamEvent, ApiError>> + Send + 'static>>

Stream messages from the LLM provider.

Sends the conversation history (messages), an optional system prompt, and optional ToolSchema definitions to the LLM, returning a 'static Stream of StreamEvents.

Called by the agent’s turn-processing loop for every LLM interaction. The stream produces events such as MessageStart, PartStart, IndexedDelta, MessageDelta, and MessageStop.

Streaming Lifetime

The returned stream is 'static — implementations must clone all required data from &self before constructing the stream. No references to &self may be captured.

Parameters

• messages — The conversation history as a Vec<Message>. Takes ownership because the request body must be built from owned data for the returned + '_ future; callers (e.g. BareLoop) clone the full history each turn — O(n) in the number of messages.
• system — An optional system prompt to prepend.
• tools — Optional tool definitions the model may invoke.

Returns

A pinned, boxed stream of Result<StreamEvent, ApiError>.

fn create_message( &self, messages: Vec<Message>, system: Option<String>, tools: Option<Vec<ToolSchema>>, ) -> Pin<Box<dyn Future<Output = Result<Value, ApiError>> + Send + '_>>

Non-streaming message request (fallback).

Sends the same parameters as stream_messages but returns a single serde_json::Value instead of a stream. Useful for simple one-shot queries where streaming overhead isn’t needed, or as a fallback when the provider does not support streaming.

Called by utility code that needs a complete response in one shot, such as token estimation probes or health checks.

Parameters

• messages — The conversation history as a Vec<Message>. Takes ownership because the request body must be built from owned data for the returned + '_ future; callers (e.g. BareLoop) clone the full history each turn — O(n) in the number of messages.
• system — An optional system prompt to prepend.
• tools — Optional tool definitions the model may invoke.

Returns

A pinned, boxed future resolving to the raw JSON response value from the provider, or an ApiError if the request fails.

Provided Methods

fn set_model(&self, _model: &str) -> bool

Attempt to switch the model at runtime.

Returns true if the client supports hot-swapping and the model was updated successfully. Returns false by default (not supported). Provider implementations that store their model behind interior mutability override this to enable BareLoop::switch_model.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors