kalosm_language_model

Trait TextCompletionModel

Source 
pub trait TextCompletionModel<Sampler = GenerationParameters>: CreateTextCompletionSession {
    // Required method
    fn stream_text_with_callback<'a>(
        &'a self,
        session: &'a mut Self::Session,
        text: &str,
        sampler: Sampler,
        on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static,
    ) -> impl Future<Output = Result<(), Self::Error>> + Send + 'a;
}
Expand description

A trait for unstructured text completion models. This trait is required for any text completion models even if they do not support structured generation. While this trait is implemented for all text completion models, most methods to use models that implement this trait are implemented in the TextCompletionModelExt trait.

§Example

use kalosm::language::*;

#[tokio::main]
async fn main() {
    // Create a new model which implements the CreateTextCompletionSession trait
    let mut llm = Llama::new().await.unwrap();
    // Create a new session for the model
    let mut session = llm.new_session().unwrap();
    // Feed some text into the session using the raw text completion api that accepts a session, prompt, sampler, and on token callback
    llm.stream_text_with_callback(&mut session, "The capital of France is ", GenerationParameters::new(), |token| {println!("{token}"); Ok(())}).await.unwrap();
}

Required Methods§

Source

fn stream_text_with_callback<'a>( &'a self, session: &'a mut Self::Session, text: &str, sampler: Sampler, on_token: impl FnMut(String) -> Result<(), Self::Error> + Send + Sync + 'static, ) -> impl Future<Output = Result<(), Self::Error>> + Send + 'a

Generate text with the given prompt.

See TextCompletionModelExt::complete for nicer API with an example.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§