onwards 0.18.1

A flexible LLM proxy library
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Onwards

[![Crates.io](https://img.shields.io/crates/v/onwards)](https://crates.io/crates/onwards)
[![Documentation](https://docs.rs/onwards/badge.svg)](https://docs.rs/onwards)
[![GitHub](https://img.shields.io/badge/GitHub-doublewordai%2Fonwards-blue)](https://github.com/doublewordai/onwards)

A Rust-based AI Gateway that provides a unified interface for routing requests to OpenAI-compatible targets. The goal is to be as "transparent" as possible.

## Features

- **Unified routing** to any OpenAI-compatible provider
- **Hot-reloading** configuration with automatic file watching
- **Authentication** with global and per-target API keys
- **Rate limiting** per-target and per-API-key with token bucket algorithm
- **Concurrency limiting** per-target and per-API-key
- **Load balancing** with weighted random and priority strategies
- **Automatic failover** across multiple providers
- **Strict mode** for request validation, response sanitization, and error standardization
- **Response sanitization** for OpenAI schema compliance
- **Prometheus metrics** for monitoring
- **Custom response headers** for pricing and metadata
- **Upstream auth customization** for non-standard providers