LargeLanguageModel

kproc_llm

Trait LargeLanguageModel 

Source 
pub trait LargeLanguageModel {
    // Required methods
    fn chat_stream(
        &self,
        prompt: ChatPrompt,
    ) -> Result<impl Future<Output = Result<StringStream>> + Send>;
    fn generate_stream(
        &self,
        prompt: GenerationPrompt,
    ) -> Result<impl Future<Output = Result<StringStream>> + Send>;

    // Provided methods
    fn chat(
        &self,
        prompt: ChatPrompt,
    ) -> Result<impl Future<Output = Result<String>> + Send> { ... }
    fn generate(
        &self,
        prompt: GenerationPrompt,
    ) -> Result<impl Future<Output = Result<String>> + Send> { ... }
}
Expand description

LLM

Required Methods§

Source

fn chat_stream( &self, prompt: ChatPrompt, ) -> Result<impl Future<Output = Result<StringStream>> + Send>

Chat with a model, returning a stream

Source

fn generate_stream( &self, prompt: GenerationPrompt, ) -> Result<impl Future<Output = Result<StringStream>> + Send>

Run inference on a model, returning a stream

Provided Methods§

Source

fn chat( &self, prompt: ChatPrompt, ) -> Result<impl Future<Output = Result<String>> + Send>

Run chat on a model, return once the complete answer has been computed. The default implementation call chat_stream until completion of the stream.

Source

fn generate( &self, prompt: GenerationPrompt, ) -> Result<impl Future<Output = Result<String>> + Send>

Run inference on a model, return once the complete answer has been computed. The default implementation call infer_stream until completion of the stream.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§