Skip to main content

LlmProvider

agent_sdk

Trait LlmProvider 

Source 
pub trait LlmProvider: Send + Sync {
    // Required methods
    fn chat<'life0, 'async_trait>(
        &'life0 self,
        request: ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ChatOutcome, Error>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             Self: 'async_trait;
    fn model(&self) -> &str;
    fn provider(&self) -> &'static str;

    // Provided methods
    fn list_models<'life0, 'async_trait>(
        &'life0 self,
    ) -> Pin<Box<dyn Future<Output = Result<Vec<ModelInfo>, Error>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             Self: 'async_trait { ... }
    fn chat_stream(
        &self,
        request: ChatRequest,
    ) -> Pin<Box<dyn Stream<Item = Result<StreamDelta, Error>> + Send + '_>> { ... }
    fn configured_thinking(&self) -> Option<&ThinkingConfig> { ... }
    fn capabilities(&self) -> Option<&'static ModelCapabilities> { ... }
    fn validate_thinking_config(
        &self,
        thinking: Option<&ThinkingConfig>,
    ) -> Result<(), Error> { ... }
    fn resolve_thinking_config(
        &self,
        request_thinking: Option<&ThinkingConfig>,
    ) -> Result<Option<ThinkingConfig>, Error> { ... }
    fn default_max_tokens(&self) -> u32 { ... }
    fn structured_output_support(&self) -> StructuredOutputSupport { ... }
}

Required Methods§

Source

fn chat<'life0, 'async_trait>( &'life0 self, request: ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatOutcome, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait, Self: 'async_trait,

Non-streaming chat completion.

Source

fn model(&self) -> &str

Source

fn provider(&self) -> &'static str

Provided Methods§

Source

fn list_models<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<ModelInfo>, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait, Self: 'async_trait,

List the models the provider currently exposes, queried live from the provider’s own model-listing endpoint.

The default implementation returns an error: model discovery is an additive capability, so a provider that has not implemented it stays source-compatible while reporting that the operation is unsupported.

§Errors

Returns an error when the provider does not support live model listing, or when the underlying HTTP request or response parsing fails.

Source

fn chat_stream( &self, request: ChatRequest, ) -> Pin<Box<dyn Stream<Item = Result<StreamDelta, Error>> + Send + '_>>

Streaming chat completion.

Returns a stream of StreamDelta events. The default implementation calls chat() and converts the result to a single-chunk stream.

Providers should override this method to provide true streaming support.

Source

fn configured_thinking(&self) -> Option<&ThinkingConfig>

Provider-owned thinking configuration, if any.

Source

fn capabilities(&self) -> Option<&'static ModelCapabilities>

Canonical capability metadata for this provider/model, if known.

Source

fn validate_thinking_config( &self, thinking: Option<&ThinkingConfig>, ) -> Result<(), Error>

Validate a thinking configuration against the provider/model capabilities.

§Errors

Returns an error when the requested thinking mode is not supported by the active provider/model capability set.

Source

fn resolve_thinking_config( &self, request_thinking: Option<&ThinkingConfig>, ) -> Result<Option<ThinkingConfig>, Error>

Resolve the effective thinking configuration for a request.

Request-level thinking overrides provider-owned defaults when present.

§Errors

Returns an error when the resolved thinking configuration is not supported by the active provider/model capability set.

Source

fn default_max_tokens(&self) -> u32

Default maximum output tokens for this provider/model when the caller does not explicitly override AgentConfig.max_tokens.

Source

fn structured_output_support(&self) -> StructuredOutputSupport

How this provider satisfies a structured-output (ResponseFormat) request.

Providers with a native JSON-schema / JSON-mode wire field (OpenAI-family, Gemini, Vertex) report StructuredOutputSupport::Native and consume request.response_format directly. Providers without one (Anthropic) report StructuredOutputSupport::ToolForcing so the runner forces a single “respond” tool whose schema is the output schema. The default is the conservative StructuredOutputSupport::ToolForcing, which works for any tool-capable provider.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§