Struct LiteLlmProvider 

pub struct LiteLlmProvider { /* private fields */ }

LiteLLM proxy provider — OpenAI-compatible, optional Bearer auth. Expects model names prefixed with “litellm:” (e.g. “litellm:gpt-4o”).

Implementations

impl LiteLlmProvider

pub fn new(base_url: impl Into<String>, api_key: impl Into<String>) -> Self

base_url is required (e.g. “http://localhost:4000”). api_key may be empty — auth header is omitted when it is.

Trait Implementations

impl LlmProvider for LiteLlmProvider

fn generate<'life0, 'async_trait>( &'life0 self, req: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate a response (non-streaming)

fn name(&self) -> &str

Provider name for routing/logging

fn supports_model(&self, model: &str) -> bool

Models supported by this provider (prefix matching)