llmshim 0.2.0

Blazing fast LLM API translation layer in pure Rust
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

# One conversation shape. The model is the routing address.

Write a conversation once:

```json
{
  "model": "openai/gpt-5.6-sol",
  "messages": [
    { "role": "user", "content": "Explain ownership in one sentence." }
  ]
}
```

To send the same conversation to another provider, change one line:

```diff
- "model": "openai/gpt-5.6-sol"
+ "model": "anthropic/claude-opus-4-8"
```

llmshim selects the provider, translates the request into that provider's native
API, sends it, and translates the result back. **Change the model, not the
conversation.** That is the central idea.

At its core, llmshim is a Rust library. The CLI and HTTP proxy wrap the same
library. The Python, TypeScript, Go, and Ruby clients are thin clients for the
proxy; Python and TypeScript can also start a bundled proxy automatically.

```mermaid
flowchart LR
    Rust[Rust application] --> Core[llmshim Rust engine]
    Terminal[Terminal user] --> CLI[llmshim chat]
    CLI --> Core
    HTTP[HTTP callers] --> Proxy[llmshim proxy]
    Clients[Language clients] --> Proxy
    Proxy --> Core
    Core --> Providers[OpenAI / Anthropic / Gemini / xAI]
```

## A translation boundary

Think of llmshim like a compiler with provider-specific backends. The input is
an OpenAI Chat Completions-style conversation. The model string selects a
backend. That backend translates the request to the provider's native API and
normalizes the provider's response.

```mermaid
flowchart LR
    Request[Model + messages + controls] --> Router[Resolve model address]
    Router --> Outbound[Translate request]
    Outbound --> API[Provider-native API]
    API --> Inbound[Translate response]
    Inbound --> Result[Normalized result]
```

Three rules define the boundary:

1. **Common features are translated where the target supports them.** Messages,
   tools, images, streaming, and reasoning controls all pass through a
   provider-specific adapter.
2. **Different APIs do not become identical.** A provider may support fewer
   reasoning levels, require another image representation, or omit a feature.
   llmshim maps, clamps, or omits fields according to the target.
3. **Native controls remain available.** Use `x-openai`, `x-anthropic`, or
   `x-gemini` when a portable control is not enough. There is no `x-xai`
   namespace.

llmshim is not an agent framework. It does not execute tools or own conversation
memory. Your application remains responsible for both.

Next, [choose the surface that fits your application](start/choose.md), or see
[how translation flows](concepts/translation-flow.md) in more detail.