Trait DynProvider 

pub trait DynProvider: Send + Sync {
    // Required methods
    fn generate_boxed<'a>(
        &'a self,
        params: &'a ChatParams,
    ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, LlmError>> + Send + 'a>>;
    fn stream_boxed<'a>(
        &'a self,
        params: &'a ChatParams,
    ) -> Pin<Box<dyn Future<Output = Result<ChatStream, LlmError>> + Send + 'a>>;
    fn metadata(&self) -> ProviderMetadata;
}

Object-safe counterpart of Provider for dynamic dispatch.

You rarely implement this directly — the blanket impl<T: Provider> DynProvider for T does it for you. Use this when you need to erase the concrete provider type:

use llm_stack::{DynProvider, ChatParams};

async fn ask(provider: &dyn DynProvider, question: &str) -> String {
    let params = ChatParams {
        messages: vec![llm_stack::ChatMessage::user(question)],
        ..Default::default()
    };
    let resp = provider.generate_boxed(&params).await.unwrap();
    format!("{resp:?}")
}

Required Methods

fn generate_boxed<'a>( &'a self, params: &'a ChatParams, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, LlmError>> + Send + 'a>>

Boxed-future version of Provider::generate.

fn stream_boxed<'a>( &'a self, params: &'a ChatParams, ) -> Pin<Box<dyn Future<Output = Result<ChatStream, LlmError>> + Send + 'a>>

Boxed-future version of Provider::stream.

fn metadata(&self) -> ProviderMetadata

Returns static metadata describing this provider instance.

Implementors

impl<T: Provider> DynProvider for T