LlmProvider

agent_core::client::traits

Trait LlmProvider 

Source 
pub trait LlmProvider {
    // Required method
    fn send_msg(
        &self,
        client: &HttpClient,
        messages: &[Message],
        options: &MessageOptions,
    ) -> Pin<Box<dyn Future<Output = Result<Message, LlmError>> + Send>>;

    // Provided method
    fn send_msg_stream(
        &self,
        _client: &HttpClient,
        _messages: &[Message],
        _options: &MessageOptions,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<StreamEvent, LlmError>> + Send>>, LlmError>> + Send>> { ... }
}
Expand description

Provider interface for LLM APIs.

Implement this trait to add support for new LLM providers.

Required Methods§

Source

fn send_msg( &self, client: &HttpClient, messages: &[Message], options: &MessageOptions, ) -> Pin<Box<dyn Future<Output = Result<Message, LlmError>> + Send>>

Send a message to the LLM. Returns the assistant’s response message or an error.

Provided Methods§

Source

fn send_msg_stream( &self, _client: &HttpClient, _messages: &[Message], _options: &MessageOptions, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<StreamEvent, LlmError>> + Send>>, LlmError>> + Send>>

Send a streaming message to the LLM. Returns a stream of events as they arrive from the API.

Implementors§