Struct LlmClient 

pub struct LlmClient {
    pub base_url: String,
    pub model: String,
    /* private fields */
}

OpenAI-compatible streaming LLM client.

Fields

base_url: String

model: String

Implementations

impl LlmClient

pub fn new( url: &str, model: &str, api_key: &str, connect_timeout_secs: u64, ) -> Self

pub async fn chat_stream( &self, messages: &[Message], ) -> Result<impl Stream<Item = Result<String>>>

POST to /v1/chat/completions with stream:true, yield content delta strings.

pub async fn auto_detect() -> Option<String>

Probe :11434 (Ollama) then :8080 (llama.cpp). Returns first base URL that responds.