llmrust 0.1.2

Unified Rust interface for 7 LLM providers (OpenAI, Anthropic, DeepSeek, Google Gemini, Ollama, Moonshot, OpenRouter) with HTTP proxy
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

# Examples


Each example is a standalone binary demonstrating a specific feature. Run with:

```bash
cargo run --example <name>
```

Some examples require API keys. Check the source for `std::env::var` calls to see which environment variables are needed.

## Index


| Example | What it demonstrates | Requires API key? |
|---------|---------------------|-------------------|
| `demo` | Register all 7 providers, call each (chat), then a stream demo | Yes (any one provider) |
| `chat` | Interactive REPL or single-prompt CLI with streaming and multi-turn support | Yes (OPENAI_API_KEY) |
| `multiturn` | 3-turn conversation with system prompt and manual history management | Yes (OPENAI_API_KEY) |
| `multimodal` | Send an image + text message (GPT-4o vision) | Yes (OPENAI_API_KEY) |
| `tool_calling` | Full tool-call loop: define tool, receive call, simulate execution, return result | Yes (OPENAI_API_KEY) |
| `retry_e2e` | End-to-end retry with `RetryProvider` and `LmrsClient::with_retry()` | Yes (any one provider) |
| `embeddings` | Generate single and batch text embeddings with `LmrsClient::embed` / `embed_batch` | Yes (OpenAI-compatible key) or local Ollama |
| `proxy_server` | Start the HTTP proxy server (dual-protocol OpenAI + Anthropic) | Yes (need `--features proxy`) |
| `router` | Multi-deployment routing with fallback and round-robin load balancing via `Router` | Yes (OPENAI_API_KEY + ANTHROPIC_API_KEY) |
| `e2e_openai_compat` | End-to-end checks against a live OpenAI-compatible endpoint (chat, stream, tools, multi-turn, image) | Yes (E2E_API_KEY + E2E_BASE_URL + E2E_MODEL) |
| `e2e_anthropic_compat` | End-to-end checks against a live Anthropic endpoint (chat, stream, image, streaming tool calls) | Yes (E2E_API_KEY + E2E_BASE_URL + E2E_MODEL) |

## Quick run


### Chat demo (needs one API key)


```bash
export OPENAI_API_KEY="sk-..."
cargo run --example demo
```

### Interactive chat


```bash
export OPENAI_API_KEY="sk-..."
cargo run --example chat -- --multi --stream
```

### Tool calling


```bash
export OPENAI_API_KEY="sk-..."
cargo run --example tool_calling
```

### Proxy server


```bash
export LLMRUST_OPENAI_KEY="sk-..."
# Optional auth

export LLMRUST_PROXY_KEY="some-shared-secret"
cargo run --example proxy_server --features proxy
```

### Embeddings (OpenAI)


```bash
export OPENAI_API_KEY="sk-..."
cargo run --example embeddings
```

### Embeddings (Ollama)


```bash
ollama pull nomic-embed-text
export LLMRUST_EMBEDDINGS_MODEL="ollama/nomic-embed-text"
cargo run --example embeddings
```

### Router (fallback + round-robin)


```bash
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
cargo run --example router
```

## Adding a new example


1. Create `examples/<your_example>.rs`
2. Add a `[[example]]` entry to `Cargo.toml`
3. Update this `README.md` index table
4. Keep the example self-contained — it should run with `cargo run --example <name>` without extra setup beyond environment variables