Skip to main content

LargeLanguageModel

kproc_llm

Trait LargeLanguageModel 

Source 
pub trait LargeLanguageModel {
    // Required methods
    fn chat_stream(
        &self,
        prompt: ChatPrompt,
    ) -> Result<impl Future<Output = Result<StringStream>> + Send>;
    fn generate_stream(
        &self,
        prompt: GenerationPrompt,
    ) -> Result<impl Future<Output = Result<StringStream>> + Send>;

    // Provided methods
    fn chat(
        &self,
        prompt: ChatPrompt,
    ) -> Result<impl Future<Output = Result<String>> + Send> { ... }
    fn generate(
        &self,
        prompt: GenerationPrompt,
    ) -> Result<impl Future<Output = Result<String>> + Send> { ... }
}
Expand description

LLM

Required Methods§

Source

fn chat_stream( &self, prompt: ChatPrompt, ) -> Result<impl Future<Output = Result<StringStream>> + Send>

Chat with a model, returning a stream

Source

fn generate_stream( &self, prompt: GenerationPrompt, ) -> Result<impl Future<Output = Result<StringStream>> + Send>

Run inference on a model, returning a stream

Provided Methods§

Source

fn chat( &self, prompt: ChatPrompt, ) -> Result<impl Future<Output = Result<String>> + Send>

Run chat on a model, return once the complete answer has been computed. The default implementation call chat_stream until completion of the stream.

Source

fn generate( &self, prompt: GenerationPrompt, ) -> Result<impl Future<Output = Result<String>> + Send>

Run inference on a model, return once the complete answer has been computed. The default implementation call infer_stream until completion of the stream.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§