kalosm_language_model

Trait StructuredChatModel

Source 
pub trait StructuredChatModel<Constraints: ModelConstraints, Sampler = GenerationParameters>: ChatModel<Sampler> {
    // Required method
    fn add_message_with_callback_and_constraints<'a>(
        &'a self,
        session: &'a mut Self::ChatSession,
        messages: &[ChatMessage],
        sampler: Sampler,
        constraints: Constraints,
        on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static,
    ) -> impl Future<Output = Result<Constraints::Output, Self::Error>> + Send + 'a;
}
Expand description

A trait for unstructured chat models that support structured generation. While this trait is implemented for all structured chat models, most methods to use models that implement this trait are implemented in the ChatModelExt trait.

§Example

use kalosm::language::*;

#[tokio::main]
async fn main() {
    // Create a new model which implements the CreateChatSession trait
    let llm = Llama::new_chat().await.unwrap();
    // Create a new chat session for the model
    let mut chat_session = llm.new_chat_session().unwrap();
    // Create a parser for your data. Any type that implements the `Parse` trait has the `new_parser` method
    let parser = i32::new_parser();
    // Add a message to the chat session with the given constraints
    let mut result: i32 = llm.add_message_with_callback_and_constraints(&mut chat_session, &[ChatMessage::new(MessageType::UserMessage, "5 + 5")], GenerationParameters::new(), parser, |token| {
        println!("{token}");
        Ok(())
    }).await.unwrap();
    println!("{result}");
}

Required Methods§

Source

fn add_message_with_callback_and_constraints<'a>( &'a self, session: &'a mut Self::ChatSession, messages: &[ChatMessage], sampler: Sampler, constraints: Constraints, on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static, ) -> impl Future<Output = Result<Constraints::Output, Self::Error>> + Send + 'a

Add messages to the chat session with a callback that is called for each token and a constraints the response must follow.

See ChatResponseBuilder::with_constraints for nicer API with examples

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§