Struct LlamaClient 

pub struct LlamaClient { /* private fields */ }

Llama.cpp server client (local or remote)

Compatible with llama.cpp’s OpenAI-compatible API server Typically runs on localhost:8080 or similar

Implementations

impl LlamaClient

pub fn new(base_url: impl Into<String>, model: impl Into<String>) -> Self

Create a new llama.cpp client

Arguments

• base_url - Base URL of llama.cpp server (e.g., “http://localhost:8080”)
• model - Model name (optional, llama.cpp usually ignores this)

pub fn with_http_client( base_url: impl Into<String>, model: impl Into<String>, http_client: HttpClient, ) -> Self

Create a new llama.cpp client with custom HTTP client Useful for configuring TLS, timeouts, etc.

pub fn localhost() -> Self

Create a client pointing to localhost:8080 (default llama.cpp port)

pub fn localhost_with_port(port: u16) -> Self

Create a client pointing to localhost with custom port

pub fn insecure(base_url: impl Into<String>, model: impl Into<String>) -> Self

Create a client with insecure HTTPS (accepts self-signed certificates) Useful for local development with HTTPS servers

pub fn localhost_insecure(port: u16) -> Self

Create localhost client with insecure HTTPS on custom port

Trait Implementations

impl ChatClient for LlamaClient

fn chat<'life0, 'async_trait>( &'life0 self, request: ChatRequest, ) -> Pin<Box<dyn Future<Output = LlmResult<ChatResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Send a chat completion request

fn chat_stream<'life0, 'async_trait>( &'life0 self, request: ChatRequest, ) -> Pin<Box<dyn Future<Output = LlmResult<TextStream>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Stream a chat completion request (yields text chunks as they arrive)

fn model(&self) -> &str

Get the model name this client uses

fn provider(&self) -> &str

Get the provider name (e.g., “openai”, “llama.cpp”)