Skip to main content

ChatProvider

machi::chat

Trait ChatProvider 

Source 
pub trait ChatProvider: Send + Sync {
    // Required methods
    fn chat<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: &'life1 ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ChatResponse>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn provider_name(&self) -> &'static str;
    fn default_model(&self) -> &str;

    // Provided methods
    fn chat_stream<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: &'life1 ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait { ... }
    fn supports_streaming(&self) -> bool { ... }
    fn supports_tools(&self) -> bool { ... }
    fn supports_vision(&self) -> bool { ... }
    fn supports_json_mode(&self) -> bool { ... }
}
Expand description

Trait for providers that support chat completions.

This is the primary trait that all LLM backends must implement to support chat-based interactions with language models.

Required Methods§

Source

fn chat<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Send a chat completion request and receive a complete response.

§Arguments
  • request - The chat request containing messages, tools, and parameters
§Returns

A ChatResponse containing the model’s response, or an error.

Source

fn provider_name(&self) -> &'static str

Get the name of this provider.

Used for error messages and logging.

Source

fn default_model(&self) -> &str

Get the default model for this provider.

Provided Methods§

Source

fn chat_stream<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Send a chat completion request and receive a streaming response.

This method returns a stream of StreamChunk values that can be processed as they arrive, enabling real-time display of generated content.

§Arguments
  • request - The chat request (the stream field will be set automatically)
§Returns

A stream of StreamChunk values, or an error if the request fails.

§Default Implementation

By default, this method returns an error indicating streaming is not supported. Providers should override this if they support streaming.

Source

fn supports_streaming(&self) -> bool

Check if this provider supports streaming.

Source

fn supports_tools(&self) -> bool

Check if this provider supports tool/function calling.

Source

fn supports_vision(&self) -> bool

Check if this provider supports vision (image inputs).

Source

fn supports_json_mode(&self) -> bool

Check if this provider supports JSON mode / structured outputs.

Implementors§