litellm-rs 0.4.16

A high-performance AI Gateway written in Rust, providing OpenAI-compatible APIs with intelligent routing, load balancing, and enterprise features
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

//! Core LLM Provider trait definitions
//!
//! Defines unified interface for all AI providers
//!
//! # Module Organization
//!
//! This module is split into three main components:
//! - `llm_provider` - The main LLMProvider trait definition
//! - `config` - ProviderConfig trait for configuration management
//! - `handle` - ProviderHandle struct for routing system integration
//!
//! # Design Principles
//!
//! 1. **Request uniformity**: All providers use the same request/response format
//! 2. **Capability driven**: Declare supported features through capabilities()
//! 3. **Provider agnostic**: Users don't need to know provider-specific details
//! 4. **Type safety**: Use associated types to ensure compile-time type safety
//! 5. **Async first**: All I/O operations are asynchronous
//! 6. **Observability**: Built-in cost calculation, latency statistics, and monitoring

// Module declarations
mod config;
mod handle;
pub mod llm_provider;

// Re-export all public types
pub use config::ProviderConfig;
pub use handle::ProviderHandle;
pub use llm_provider::trait_definition::LLMProvider;