Struct LocalProvider 

pub struct LocalProvider { /* private fields */ }

Local LLM provider for Ollama, LM Studio, vLLM, and other OpenAI-compatible local servers.

This provider wraps OpenAiProvider with sensible defaults for local inference:

• No API key required (empty string by default)
• Default localhost URL (Ollama: 11434)
• Extended timeout for slower local inference

Implementations

impl LocalProvider

pub const DEFAULT_OLLAMA_URL: &'static str = "http://localhost:11434/v1/chat/completions"

Default Ollama endpoint

pub const DEFAULT_LMSTUDIO_URL: &'static str = "http://localhost:1234/v1/chat/completions"

Default LM Studio endpoint

pub const DEFAULT_VLLM_URL: &'static str = "http://localhost:8000/v1/chat/completions"

Default vLLM endpoint

pub fn new( base_url: Option<&str>, model: &str, max_tokens: u32, timeout: u64, ) -> Self

Create a new local provider

Arguments

• base_url - Optional custom endpoint. Defaults to Ollama.
• model - Model name to use (required)
• max_tokens - Maximum output tokens
• timeout - Request timeout in seconds (default: 120 for slower local inference)

pub fn ollama(model: &str, max_tokens: u32, timeout: u64) -> Self

Create provider configured for Ollama

pub fn lmstudio(model: &str, max_tokens: u32, timeout: u64) -> Self

Create provider configured for LM Studio

pub fn vllm(model: &str, max_tokens: u32, timeout: u64) -> Self

Create provider configured for vLLM

Trait Implementations

impl Clone for LocalProvider

fn clone(&self) -> LocalProvider

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl LlmProvider for LocalProvider

fn send<'life0, 'async_trait>( &'life0 self, messages: Vec<Message>, tools: Vec<Tool>, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<ProviderResponseChunk, LlmError>> + Send + '_>>, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Send messages to the LLM and receive streaming response

fn provider_name(&self) -> &str

Get provider name

fn model_name(&self) -> &str

Get model name

fn clone_box(&self) -> Box<dyn LlmProvider>

Clone the provider