Trait StructuredTextCompletionModel

Source

pub trait StructuredTextCompletionModel<Constraints: ModelConstraints, Sampler = GenerationParameters>: TextCompletionModel<Sampler> {
    // Required method
    fn stream_text_with_callback_and_parser<'a>(
        &'a self,
        session: &'a mut Self::Session,
        text: &str,
        sampler: Sampler,
        parser: Constraints,
        on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static,
    ) -> impl Future<Output = Result<Constraints::Output, Self::Error>> + Send + 'a;
}

Expand description

A trait for text completion models that support structured generation. While this trait is implemented for all structured text completion models, most methods to use models that implement this trait are implemented in the TextCompletionModelExt trait.

§Example

use kalosm::language::*;

#[tokio::main]
async fn main() {
    // Create a new model which implements the CreateTextCompletionSession trait
    let mut llm = Llama::new().await.unwrap();
    // Create a new session for the model
    let mut session = llm.new_session().unwrap();
    // Create a parser for your data. Different models accept different types of parsers. The Llama model accepts
    // any parsers that implements the `Parse` trait.
    let parser = i32::new_parser();
    // Feed some text into the session using the raw structured text completion api that accepts a session, prompt, sampler, and on token callback
    llm.stream_text_with_callback_and_parser(&mut session, "5 * 5 = ", GenerationParameters::new(), parser, |token| {println!("{token}"); Ok(())}).await.unwrap();
}

Required Methods§

Source

fn stream_text_with_callback_and_parser<'a>( &'a self, session: &'a mut Self::Session, text: &str, sampler: Sampler, parser: Constraints, on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static, ) -> impl Future<Output = Result<Constraints::Output, Self::Error>> + Send + 'a

Generate text with the given prompt and the given constraints.

See TextCompletionModelExt::complete for nicer API with an example.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§

Source §