modelbridge 0.5.2

One provider-neutral Rust client for OpenAI, Anthropic, Gemini, Groq, Ollama and friends: streaming, tool calling, an agent loop, usage accounting, and subscription sign-in
Documentation
# modelbridge

Modelbridge is one small Rust API for the language-model providers people actually use. Pick a provider and a model; the crate already knows the endpoint, the credential variable, the request shape, the streaming protocol, the tool-call format, and the token prices. It is synchronous and runtime-free — no Tokio, no async colouring — so a CLI, TUI, or desktop app can run a stream on a worker thread and drain a channel each frame. On top of the transport it ships the two things applications otherwise rewrite badly: a tool-calling agent loop, and browser sign-in for the ChatGPT and Claude subscription routes.

## Layout

- `src/catalog.rs` — built-in providers and a curated, priced model list. Never an allow-list: any model id works, including one released tomorrow.
- `src/client.rs` — the HTTP client: one route, one credential, retries, and a usage meter.
- `src/encode.rs` — one request translated into Responses, Anthropic Messages, or Chat Completions, tool round trips included.
- `src/stream.rs` — the three server-sent-event dialects decoded into `Chunk`s.
- `src/agent.rs` — the call-tools-until-answered loop.
- `src/oauth/` — PKCE, the loopback redirect catcher, token storage, and refresh for the subscription routes.
- `src/meter.rs` — token and cost accounting, in total and by `provider/model`.

## Install

```toml
[dependencies]
modelbridge = "0.4"
```

Skip the subscription sign-in machinery, and its three dependencies, with `default-features = false`.

## Quickstart

```rust
use modelbridge::{Client, Request};

// Reads ANTHROPIC_API_KEY. Try "openai", "groq", "gemini", "ollama", …
let client = Client::default_for("anthropic")?;

let answer = client.complete(
    &Request::new("Be concise and practical.")
        .user("Why is the sky blue?")
        .max_output_tokens(300),
)?;

println!("{}", answer.text);
println!("{} tokens, about ${:.4}", answer.usage.total(), answer.usage.dollars());
```

## Examples

Each one runs against any provider you have a key for — pass its id as the first argument.

```sh
cargo run --example basic     -- groq        # one completion, with cost
cargo run --example streaming -- anthropic   # worker thread, channel, cancellation
cargo run --example tools     -- openai      # a tool round trip by hand
cargo run --example agent     -- anthropic   # the same thing, driven by the agent loop
cargo run --example signin    -- claude      # subscription sign-in, start to finish
```

`streaming` is the one worth reading first if you are building a TUI: it is the whole pattern — block on a worker, drain a channel, raise a flag to stop.

## Providers

| Provider | API | Credential |
|---|---|---|
| OpenAI | Responses | `OPENAI_API_KEY` |
| Anthropic | Messages | `ANTHROPIC_API_KEY` |
| OpenRouter | Chat Completions | `OPENROUTER_API_KEY` |
| Groq | Chat Completions | `GROQ_API_KEY` |
| DeepSeek | Chat Completions | `DEEPSEEK_API_KEY` |
| Google Gemini | OpenAI compatibility | `GEMINI_API_KEY` |
| xAI | Chat Completions | `XAI_API_KEY` |
| Mistral | Chat Completions | `MISTRAL_API_KEY` |
| Ollama | Local Chat Completions | none |
| ChatGPT, Claude | subscription sign-in | browser, see below |

Anything else OpenAI-shaped works without being in the table:

```rust
let client = modelbridge::Client::compatible(
    "https://models.internal.example/v1/chat/completions",
    "company-model-v2",
    std::env::var("COMPANY_MODEL_KEY").unwrap(),
)?;
```

Use `ClientBuilder::custom` for a Responses- or Anthropic-shaped endpoint. If a genuinely different wire protocol appears, implement the four-method `Provider` trait and nothing above it changes.

## Streaming and tools

`Client` implements `Provider`. Call `stream` with an `mpsc::Sender<Chunk>` to receive text deltas, normalised tool calls, terminal usage, and errors as they arrive; `complete` consumes that stream into a `Response`. Tool schemas are translated to Responses functions, Anthropic `input_schema` tools, or Chat Completions functions, and results are fed back as `function_call_output`, `tool_result` blocks, or `tool` messages — with call ids matched, which is what every provider rejects a conversation for getting wrong.

`Request::cancel` takes a flag that stops a stream at the next event boundary, so a user who hits escape is not waiting on a 180-second read timeout.

## The agent loop

```rust
use modelbridge::agent::{Agent, Step, Tools};

let mut agent = Agent::new(&client, my_tools)
    .instructions("Search before answering anything time-sensitive.")
    .max_turns(6);

let answer = agent.ask("What shipped in Rust 1.90?", &mut |step| {
    if let Step::Token(text) = step { print!("{text}") }
})?;
```

`Tools` is two methods: what the tools are, and how to run one. The loop owns the rest — replaying calls and results as real conversation turns, refusing a call the model has already made verbatim, reporting a failed tool as a failure rather than as data, and forcing an answer when the turn budget runs out. `Step::Planning` marks the text a turn streamed before deciding to call tools, so an interface can clear a paragraph that was never the answer.

## Sign-in and OAuth

API keys stay the simplest path. For a gateway that issues tokens, pass one:

```rust
let client = modelbridge::Client::builder("openai", "gpt-5.6-terra")?
    .bearer_token(access_token)
    .build()?;
```

Implement `OAuthTokenProvider` for a credential that can be refreshed; Modelbridge asks for a token before every request and, on a 401 or 403, refreshes once and retries.

The `oauth` module implements that trait for the two consumer subscription routes, including the browser flow, the loopback redirect, the credentials file, and refresh throttling:

```rust
use modelbridge::oauth::{Store, Subscription};

let store = Store::at(config_dir.join("credentials.json"));
let tokens = subscription.login("My App").run(&store, &announce, &cancel)?;
let client = subscription
    .client(subscription.session(&store, tokens), "claude-sonnet-5")
    .build()?;
```

Both subscription routes authenticate as a vendor's own first-party CLI client against an endpoint scoped to it. That is off-label use: it can stop working without notice, and it is not what a consumer subscription is sold for. The supported path for each is the ordinary API key, and nothing above `Client` can tell the two apart.

## Design

- No async runtime, and no dependency that pulls one in.
- Errors are a matchable `Error`, not a string, so retry policy and credential refresh key off the difference between a rejected key, a rate limit, and a dead connection. `Error` converts into `String` with `?`, so applications built that way lose nothing.
- Rate limits and transient server faults are retried with backoff, honouring `Retry-After`; rejected credentials and malformed requests are not.
- All three SSE decoders tolerate unknown future event types and always terminate the stream exactly once.
- Usage is tracked globally and per `provider/model`, with catalog prices producing an estimate and a provider-reported charge winning when there is one.
- Vendor errors are surfaced with their human-readable message and without echoing credentials.
- Every protocol claim above is covered by a test, including real local HTTP round trips for streaming, rate-limit retry, and OAuth refresh, without contacting a vendor.

Modelbridge began as the provider layer inside a language-learning CLI, and is now the model layer for that and a terminal research agent. Adopting it deleted about 6,300 lines from the two of them.

Created by [`Ryan Sayer`](https://github.com/Ryan-Sayer)

Release history is in [`CHANGELOG.md`](CHANGELOG.md). Licensed under [MIT](LICENSE).