Struct LlmClient 

pub struct LlmClient {
    pub model_id: String,
    /* private fields */
}

Fields

model_id: String

Implementations

impl LlmClient

pub fn new(full_model_string: &str) -> Result<Self, AicoError>

pub fn get_extra_params(&self) -> Option<Value>

pub async fn stream_chat( &self, req: ChatCompletionRequest, ) -> Result<Response, AicoError>

Sends a streaming request and returns a channel or iterator of chunks. For simplicity with ‘minimal deps’, we return the response and let the caller iterate.

Trait Implementations

impl Debug for LlmClient

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.