Skip to main content

ChatModel

juncture

Trait ChatModel 

Source 
pub trait ChatModel:
    Send
    + Sync
    + Clone
    + 'static {
    // Required methods
    fn invoke<'life0, 'life1, 'life2, 'async_trait>(
        &'life0 self,
        messages: &'life1 [Message],
        options: Option<&'life2 CallOptions>,
    ) -> Pin<Box<dyn Future<Output = Result<Message, LlmError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             Self: 'async_trait;
    fn stream<'life0, 'life1, 'life2, 'async_trait>(
        &'life0 self,
        messages: &'life1 [Message],
        options: Option<&'life2 CallOptions>,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<MessageChunk, LlmError>> + Send + 'life0>>, LlmError>> + Send + 'async_trait>>
       where 'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             Self: 'async_trait;
    fn bind_tools(&self, tools: Vec<ToolDefinition>) -> Self;
    fn with_structured_output<T>(self) -> StructuredOutputModel<Self, T>
       where T: JsonSchema + DeserializeOwned + Serialize,
             Self: Sized;
    fn model_name(&self) -> &str;
}
Expand description

Unified ChatModel trait for all LLM providers

This trait provides a common interface for interacting with different LLM providers (Anthropic, OpenAI, Ollama, etc.).

§Type Parameters

  • 'a - Lifetime for borrowed data in streaming

Required Methods§

Source

fn invoke<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, messages: &'life1 [Message], options: Option<&'life2 CallOptions>, ) -> Pin<Box<dyn Future<Output = Result<Message, LlmError>> + Send + 'async_trait>>
where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, Self: 'async_trait,

Invoke the model with messages

§Arguments
  • messages - Conversation history
  • options - Optional call settings to override defaults
§Returns

The model’s response as a complete message

Source

fn stream<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, messages: &'life1 [Message], options: Option<&'life2 CallOptions>, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<MessageChunk, LlmError>> + Send + 'life0>>, LlmError>> + Send + 'async_trait>>
where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, Self: 'async_trait,

Stream the model’s response

§Arguments
  • messages - Conversation history
  • options - Optional call settings to override defaults
§Returns

A stream of message chunks that must be accumulated

Source

fn bind_tools(&self, tools: Vec<ToolDefinition>) -> Self

Bind tools to this model instance

Returns a new instance with the tools registered for function calling.

§Arguments
  • tools - List of tool definitions
Source

fn with_structured_output<T>(self) -> StructuredOutputModel<Self, T>
where T: JsonSchema + DeserializeOwned + Serialize, Self: Sized,

Convert to structured output model

Returns a wrapper that forces the model to output structured JSON matching type T’s schema.

§Type Parameters
  • T - Target type with JSON Schema support
Source

fn model_name(&self) -> &str

Get the model name

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§