chronicle_proxy::providers

Trait ChatModelProvider

Source 
pub trait ChatModelProvider:
    Debug
    + Send
    + Sync {
    // Required methods
    fn name(&self) -> &str;
    fn label(&self) -> &str;
    fn send_request<'life0, 'async_trait>(
        &'life0 self,
        options: SendRequestOptions,
        chunk_tx: StreamingResponseSender,
    ) -> Pin<Box<dyn Future<Output = Result<(), Report<ProviderError>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
    fn is_default_for_model(&self, model: &str) -> bool;
}

Required Methods§

Source

fn name(&self) -> &str

Internal name for the provider

Source

fn label(&self) -> &str

A readable name for the provider

Source

fn send_request<'life0, 'async_trait>( &'life0 self, options: SendRequestOptions, chunk_tx: StreamingResponseSender, ) -> Pin<Box<dyn Future<Output = Result<(), Report<ProviderError>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Send a request and return the response. If there’s any chance of retryable failures for this provider (e.g. almost every provider), then this function should handle retrying with the behavior specified in options.retry. The request_with_retry function can assist with that.

Source

fn is_default_for_model(&self, model: &str) -> bool

Implementors§