Skip to main content

ChatModel

studio_worker::host

Trait ChatModel 

Source 
pub trait ChatModel {
    // Required method
    fn chat(
        &self,
        params: LlmParams,
        cancelled: &dyn Fn() -> bool,
    ) -> Result<Value>;
}
Expand description

A loaded model that answers chat completions.

Required Methods§

Source

fn chat(&self, params: LlmParams, cancelled: &dyn Fn() -> bool) -> Result<Value>

Run one completion; cancelled turns true when an unload starts. Returns OpenAI chat.completion-shaped JSON.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§