LiteLLM-RS

A high-performance AI gateway written in Rust that provides OpenAI-compatible APIs with intelligent routing across 100+ providers.

Features

• OpenAI-compatible HTTP APIs (chat, completions, embeddings, images)
• Multi-provider routing, load balancing, and failover
• Streaming, tool/function calling, and response formatting
• Caching, authentication (API keys/JWT), rate limiting, and observability
• Extensible provider system and protocol gateways (MCP, A2A)

Getting Started

• Documentation: see docs/README.md
• Configuration: copy templates from config/*.example or config/*.template
• Run locally: make dev or cargo run

Library Usage

The crate exposes Rust helpers for OpenAI-style calls. For end-to-end usage, run the gateway binary and configure providers via the config file.

License

MIT. See LICENSE.