ollama-proxy 0.1.0

Proxy server for Ollama
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131

# Ollama API Proxy Server

[![Rust CI](https://github.com/hehaoqian/ollama-proxy/actions/workflows/rust-ci.yml/badge.svg)](https://github.com/hehaoqian/ollama-proxy/actions/workflows/rust-ci.yml)
[![Security Audit](https://github.com/hehaoqian/ollama-proxy/actions/workflows/security-audit.yml/badge.svg)](https://github.com/hehaoqian/ollama-proxy/actions/workflows/security-audit.yml)

WARNING: This crate is in early development. Use at your own risk. Use in production is not recommended.

This is a lightweight HTTP server that acts as a proxy for the Ollama API. It forwards requests to an Ollama server and provides additional features like API authentication, IP allowlisting, and HTTPS support.

## Features

- **API Forwarding:** Forwards requests to an Ollama API server
- **Authentication:** Optional API key authentication for model management endpoints
- **IP Allowlisting:** Restrict access to specific IP addresses
- **HTTPS Support:** Optional TLS/SSL encryption for secure connections
- **Logging:** Console and file logging options
- **Database Logging:** Optional request and response logging to SQLite database

## Usage

### Basic HTTP Server

```bash
cargo run -- --port 3001 --ollama-url http://localhost:11434
```

### With Authentication

```bash
cargo run -- --port 3001 --ollama-url http://localhost:11434 --api-key YOUR_SECRET_KEY
```

### With IP Allowlisting

```bash
cargo run -- --port 3001 --ollama-url http://localhost:11434 --allowed-ips "127.0.0.1,192.168.1.5"
```

### With Database Logging

```bash
cargo run -- --port 3001 --ollama-url http://localhost:11434 --db-url "sqlite:logs.db"
```

### With HTTPS (Secure Connections)

First, generate a self-signed certificate for testing:

```bash
./generate_cert.sh
```

Then run the server with HTTPS enabled:

```bash
cargo run -- --https --cert-file ./certs/server.crt --key-file ./certs/server.key
```

To make the server publicly accessible on all network interfaces:

```bash
cargo run -- --https --cert-file ./certs/server.crt --key-file ./certs/server.key --host 0.0.0.0
```

### Using Environment Variables

All command-line arguments can also be specified using environment variables with the `PROXY_OLLAMA_` prefix. For example:

```bash
export PROXY_OLLAMA_PORT=3001
export PROXY_OLLAMA_URL="http://localhost:11434"
export PROXY_OLLAMA_API_KEY="your-secret-key"
export PROXY_OLLAMA_ALLOWED_IPS="127.0.0.1,192.168.1.5"
export PROXY_OLLAMA_HTTPS=true
export PROXY_OLLAMA_CERT_FILE="./certs/server.crt"
export PROXY_OLLAMA_KEY_FILE="./certs/server.key"
export PROXY_OLLAMA_HOST="0.0.0.0"
export PROXY_OLLAMA_LOG_FILE="./proxy.log"
export PROXY_OLLAMA_LOG_ROTATE_SIZE="10MB"
export PROXY_OLLAMA_MAX_LOG_FILES=5
export PROXY_OLLAMA_DB_URL="sqlite:logs.db"

cargo run
```

## Command-Line Options

| Option | Description |
|--------|-------------|
| `--port` | Port to listen on (default: 3001) |
| `--ollama-url` | Ollama server URL (default: <http://localhost:11434>) |
| `--log-file` | Output log file (if not specified, logs go to stdout only) |
| `--api-key` | API key required for model management endpoints |
| `--allowed-ips` | List of allowed IP addresses (comma-separated) |
| `--https` | Enable HTTPS mode |
| `--cert-file` | TLS certificate file path (required when HTTPS is enabled) |
| `--key-file` | TLS private key file path (required when HTTPS is enabled) |
| `--host` | Host address to listen on (default: 127.0.0.1, use 0.0.0.0 to listen on all interfaces) |
| `--log-rotate-size` | Maximum log file size before rotation (default: 10MB) |
| `--max-log-files` | Maximum number of rotated log files to keep (default: 0, unlimited) |
| `--db-url` | Database URL for request/response logging in SQLite format (e.g., "sqlite:logs.db") |

## API Endpoints

Documentation for all API endpoints is available at the root URL (e.g., `http://localhost:3001/` or `https://localhost:3001/`).

## Database Logging

The server can log all API requests and responses to a SQLite database for auditing and debugging purposes. This feature is enabled by default but requires a database URL to be specified using the `--db-url` option.

### Database Schema

When enabled, the following information is logged:

- Timestamp of the request
- Client IP address
- HTTP method
- Request path
- Request headers (as JSON)
- Request body (as JSON when possible)
- Response status code
- Response headers (as JSON)
- Response body (as JSON when possible)

### Disabling Database Logging

Database logging is a compiled feature that can be disabled at build time by excluding the "database-logging" feature:

```bash
cargo build --no-default-features
```