Skip to main content

LlmProvider

agent_base::llm

Trait LlmProvider 

Source 
pub trait LlmProvider: Send + Sync {
    // Required methods
    fn stream<'life0, 'async_trait>(
        &'life0 self,
        request: ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ChatStream, LlmError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             Self: 'async_trait;
    fn chat<'life0, 'async_trait>(
        &'life0 self,
        request: ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, LlmError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             Self: 'async_trait;
    fn capabilities(&self) -> Capabilities;
    fn info(&self) -> ProviderInfo;
}
Expand description

LLM Provider unified interface.

The core trait exposed by the framework. All providers implement this. Supports both streaming and non-streaming call modes.

§Object Safety

This trait is object-safe, so Arc<dyn LlmProvider> works.

Required Methods§

Source

fn stream<'life0, 'async_trait>( &'life0 self, request: ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatStream, LlmError>> + Send + 'async_trait>>
where 'life0: 'async_trait, Self: 'async_trait,

Streaming call (returns chunks in real-time).

Use for: chat interfaces, real-time output, long text generation.

Source

fn chat<'life0, 'async_trait>( &'life0 self, request: ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, LlmError>> + Send + 'async_trait>>
where 'life0: 'async_trait, Self: 'async_trait,

Non-streaming call (returns complete result at once).

Use for: API services, batch processing, testing.

Source

fn capabilities(&self) -> Capabilities

Get provider capabilities.

Source

fn info(&self) -> ProviderInfo

Get provider info.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§