Trait LlmProvider 

pub trait LlmProvider: Send + Sync {
    // Required methods
    fn info(&self) -> ProviderInfo;
    fn capabilities(&self) -> Capabilities;
    fn list_models(
        &self,
    ) -> Pin<Box<dyn Future<Output = Result<Vec<ModelInfo>, ProviderError>> + Send + '_>>;
    fn model_info(&self, model_id: &str) -> Option<ModelInfo>;
    fn complete(
        &self,
        req: CompletionRequest,
        cancel: CancellationToken,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<ProviderChunk, ProviderError>> + Send>>, ProviderError>> + Send + '_>>;

    // Provided method
    fn hosted_capabilities(&self) -> HostedCapabilities { ... }
}

LLM provider abstraction.

Cancellation semantics: LlmProvider::complete receives a CancellationToken; the caller may call cancel() at any point to abort the call and the downstream stream. Dropping the returned stream also counts as cancellation.

Required Methods

fn info(&self) -> ProviderInfo

Provider metadata (vendor name, API style, tracing labels, etc.).

fn capabilities(&self) -> Capabilities

Provider-level capability matrix. Model-level differences are expressed via super::ModelCapabilityOverrides and merged on demand by the main loop.

fn list_models( &self, ) -> Pin<Box<dyn Future<Output = Result<Vec<ModelInfo>, ProviderError>> + Send + '_>>

Lists the models currently available from this provider.

The implementation may make network calls (e.g., OpenAI /v1/models); results should be cached inside the provider for synchronous lookup by Self::model_info.

Errors

Network errors, authentication errors, server errors, etc., are all mapped to ProviderError.

fn model_info(&self, model_id: &str) -> Option<ModelInfo>

Synchronously query metadata for a given model.

This is a fast path for context trimming in the main loop; must not trigger a network call. Returns None if the provider’s cache does not contain the model. The caller may then decide to call Self::list_models and retry, or treat it as an unknown model.

fn complete( &self, req: CompletionRequest, cancel: CancellationToken, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<ProviderChunk, ProviderError>> + Send>>, ProviderError>> + Send + '_>>

Start a streaming generation.

Errors

Authentication failures, invalid parameters, transport errors, server errors, etc. are all mapped to ProviderError. Errors produced within the stream are delivered via the stream’s Err variant, not through this return value.

Provided Methods

fn hosted_capabilities(&self) -> HostedCapabilities

The set of hosted capabilities that this provider adapter advertises.

Unlike Self::capabilities (which describes model-level properties), this reflects the current adapter implementation state: whether it can expose hosted web_search, fetch, etc. to the model over the wire. During session startup, this value is read together with capabilities.web_search.mode to determine the source of hosted web search capabilities.

The default implementation returns all false; new providers do not need to override it. Adapters that truly support hosted capabilities (Anthropic / OpenAI Responses) should explicitly override this method.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors