litellm-rs 0.4.16

A high-performance AI Gateway written in Rust, providing OpenAI-compatible APIs with intelligent routing, load balancing, and enterprise features
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

//! Core router for AI provider management and request routing
//!
//! This module provides intelligent routing, load balancing, and failover
//! across multiple AI providers.
//!
//! ## Module Structure
//!
//! The router is organized into modular components following the single-responsibility principle:
//!
//! - `config` - Router configuration and routing strategy definitions
//! - `error` - Error types and cooldown reasons
//! - `fallback` - Fallback configuration and execution results
//! - `deployment` - Deployment management and health tracking
//! - `unified` - Core Router struct and deployment management
//! - `selection` - Deployment selection logic
//! - `strategy_impl` - Routing strategy implementations
//! - `execution` - Execution helpers and error conversion
//! - `execute_impl` - Execute methods with retry and fallback support
//! - `gateway_config` - Gateway configuration integration

// New modular router components
pub mod budget_routing;
pub mod config;
pub mod deployment;
pub mod error;
pub mod execute_impl;
pub mod execution;
pub mod fallback;
pub mod gateway_config;
pub mod selection;
pub mod strategy_impl;
pub mod unified;

#[cfg(test)]
mod tests;

// Re-exports from deployment module
pub use deployment::{Deployment, DeploymentConfig, DeploymentId, DeploymentState, HealthStatus};

// Re-exports from new modular router (UnifiedRouter)
pub use budget_routing::{BudgetAwareRouter, BudgetAwareRouting, RequestBudgetCheck};
pub use config::{RouterConfig, RoutingStrategy as UnifiedRoutingStrategy};
pub use error::{CooldownReason, RouterError};
pub use fallback::{ExecutionResult, FallbackConfig, FallbackType};
pub use unified::{Router as UnifiedRouter, RoutingMetrics};