Trait ChatDriver

Source

pub trait ChatDriver: Send + Sync {
    // Required methods
    fn id(&self) -> DriverId;
    fn complete<'life0, 'async_trait>(
        &'life0 self,
        request: ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, Error>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             Self: 'async_trait;

    // Provided method
    fn complete_streaming<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: ChatRequest,
        sink: &'life1 mut dyn DeltaSink,
    ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, Error>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             'life1: 'async_trait,
             Self: 'async_trait { ... }
}

Expand description

One provider protocol. Implementations translate ChatRequest to the provider’s wire format and back.

Required Methods§

Source

fn id(&self) -> DriverId

The protocol id this driver answers to; a ModelSpec naming it routes here.

Source

fn complete<'life0, 'async_trait>( &'life0 self, request: ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait, Self: 'async_trait,

Produce the next assistant message.

Errors should be crate::error::Error::Driver with a crate::error::LlmErrorKind, so a host can tell a transient failure from a terminal one.

Provided Methods§

Source

fn complete_streaming<'life0, 'life1, 'async_trait>( &'life0 self, request: ChatRequest, sink: &'life1 mut dyn DeltaSink, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Stream a completion, reporting text as it arrives through sink, and return the same final ChatResponse complete would. The default forwards to complete and reports the whole response as a single delta — real incremental streaming is an opt-in per-driver upgrade (see the http feature’s OpenAI/Anthropic drivers), not a requirement of the trait.