Trait Provider 

pub trait Provider: Send + Sync {
    // Required methods
    fn name(&self) -> &str;
    fn api(&self) -> &str;
    fn model_id(&self) -> &str;
    fn stream<'life0, 'life1, 'life2, 'life3, 'async_trait>(
        &'life0 self,
        context: &'life1 Context<'life2>,
        options: &'life3 StreamOptions,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<StreamEvent>> + Send>>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait;
}

An LLM backend capable of streaming assistant output (and tool calls).

A Provider is typically configured for a specific API + model and is used by the agent loop to produce a stream of StreamEvent updates.

Required Methods

fn name(&self) -> &str

Get the provider name.

fn api(&self) -> &str

Get the API type.

fn model_id(&self) -> &str

Get the model identifier used by this provider.

fn stream<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, context: &'life1 Context<'life2>, options: &'life3 StreamOptions, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<StreamEvent>> + Send>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Start streaming a completion.

Implementations should yield StreamEvent items as soon as they are decoded, and should stop promptly when the request is cancelled.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors