kalosm_language_model

Trait ChatModel

Source 
pub trait ChatModel<Sampler = GenerationParameters>: CreateChatSession {
    // Required method
    fn add_messages_with_callback<'a>(
        &'a self,
        session: &'a mut Self::ChatSession,
        messages: &[ChatMessage],
        sampler: Sampler,
        on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static,
    ) -> impl Future<Output = Result<(), Self::Error>> + Send + 'a;
}
Expand description

A trait for unstructured chat models. This trait is required for any chat models even if they do not support structured generation. While this trait is implemented for all chat models, most methods to use models that implement this trait are implemented in the ChatModelExt trait.

§Example

use kalosm::language::*;

#[tokio::main]
async fn main() {
    // Create a new model which implements the CreateChatSession trait
    let llm = Llama::new_chat().await.unwrap();
    // Create a new chat session for the model
    let mut chat_session = llm.new_chat_session().unwrap();
    // Add a message to the chat session
    llm.add_messages_with_callback(
        &mut chat_session,
        &[ChatMessage::new(MessageType::UserMessage, "Hello, world!")],
        GenerationParameters::new(),
        |token| {
            println!("{token}");
            Ok(())
        },
    )
    .await
    .unwrap();
}

Required Methods§

Source

fn add_messages_with_callback<'a>( &'a self, session: &'a mut Self::ChatSession, messages: &[ChatMessage], sampler: Sampler, on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static, ) -> impl Future<Output = Result<(), Self::Error>> + Send + 'a

Add messages to the chat session with a callback that is called for each token.

See Chat::add_message for nicer API with examples

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§