Trait Provider 

pub trait Provider: Send + Sync {
    // Required methods
    fn generate(
        &self,
        params: &ChatParams,
    ) -> impl Future<Output = Result<ChatResponse, LlmError>> + Send;
    fn stream(
        &self,
        params: &ChatParams,
    ) -> impl Future<Output = Result<ChatStream, LlmError>> + Send;
    fn metadata(&self) -> ProviderMetadata;
}

The core trait every LLM provider implements.

Provider uses native async-fn-in-traits (Rust 2024 edition). Implementations are plain async fns — no #[async_trait] needed.

Cross-cutting concerns like retries, rate-limiting, and logging are handled by the interceptor system, keeping individual backends focused on HTTP mapping.

Object safety

Provider is not object-safe because AFIT returns impl Future. When you need dynamic dispatch (e.g. Box<dyn _> or Arc<dyn _>), use DynProvider instead — every Provider automatically implements DynProvider via a blanket impl.

Required Methods

fn generate( &self, params: &ChatParams, ) -> impl Future<Output = Result<ChatResponse, LlmError>> + Send

Sends a chat completion request and returns the full response.

fn stream( &self, params: &ChatParams, ) -> impl Future<Output = Result<ChatStream, LlmError>> + Send

Sends a chat completion request and returns a stream of events.

The returned ChatStream yields StreamEvents as they arrive from the provider.

fn metadata(&self) -> ProviderMetadata

Returns static metadata describing this provider instance.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors