litellm-rs 0.4.16

A high-performance AI Gateway written in Rust, providing OpenAI-compatible APIs with intelligent routing, load balancing, and enterprise features
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109

# 📜 Deployment Scripts

## Available Scripts

### 🚀 `start.sh`
Quick start script for local development.

```bash
./deployment/scripts/start.sh
```

**What it does:**
- Checks for `config/gateway.yaml`
- Compiles and runs the gateway
- Shows helpful error messages

### 🔧 `setup.sh`
Environment setup script.

```bash
./deployment/scripts/setup.sh
```

**What it does:**
- Installs system dependencies
- Sets up database
- Creates configuration files

### 🐳 `docker-start.sh`
Docker container startup script.

```bash
./deployment/scripts/docker-start.sh
```

**What it does:**
- Runs inside Docker container
- Handles environment setup
- Starts the gateway service

### 🗄️ Database Scripts

#### `init-db.sql`
Production database initialization.

```bash
psql -f deployment/scripts/init-db.sql
```

#### `init-dev-db.sql`
Development database setup.

```bash
psql -f deployment/scripts/init-dev-db.sql
```

### 🧪 `test_api.sh`
API testing script.

```bash
./deployment/scripts/test_api.sh
```

**What it does:**
- Tests all API endpoints
- Validates responses
- Reports results

## 🔧 Script Configuration

### Environment Variables
```bash
# Set these before running scripts
export GATEWAY_HOST=0.0.0.0
export GATEWAY_PORT=8000
export DATABASE_URL=postgresql://...
```

### Custom Paths
```bash
# Override default paths
export CONFIG_PATH=custom/config.yaml
export LOG_PATH=custom/logs/
```

## 📋 Usage Examples

### Development Workflow
```bash
# 1. Setup environment
./deployment/scripts/setup.sh

# 2. Start gateway
./deployment/scripts/start.sh

# 3. Test API
./deployment/scripts/test_api.sh
```

### Production Deployment
```bash
# 1. Setup production environment
sudo ./deployment/scripts/setup.sh --production

# 2. Install systemd service
sudo cp ../systemd/litellm-rs.service /etc/systemd/system/
sudo systemctl enable litellm-rs
sudo systemctl start litellm-rs
```