modelbridge
Modelbridge is one small Rust API for the language-model providers people actually use. Pick a provider and a model; the crate already knows the endpoint, the credential variable, the request shape, the streaming protocol, the tool-call format, and the token prices. It is synchronous and runtime-free — no Tokio, no async colouring — so a CLI, TUI, or desktop app can run a stream on a worker thread and drain a channel each frame. On top of the transport it ships the two things applications otherwise rewrite badly: a tool-calling agent loop, and browser sign-in for the ChatGPT and Claude subscription routes.
Layout
src/catalog.rs— built-in providers and a curated, priced model list. Never an allow-list: any model id works, including one released tomorrow.src/client.rs— the HTTP client: one route, one credential, retries, and a usage meter.src/encode.rs— one request translated into Responses, Anthropic Messages, or Chat Completions, tool round trips included.src/stream.rs— the three server-sent-event dialects decoded intoChunks.src/agent.rs— the call-tools-until-answered loop.src/oauth/— PKCE, the loopback redirect catcher, token storage, and refresh for the subscription routes.src/meter.rs— token and cost accounting, in total and byprovider/model.
Install
[]
= "0.4"
Skip the subscription sign-in machinery, and its three dependencies, with default-features = false.
Quickstart
use ;
// Reads ANTHROPIC_API_KEY. Try "openai", "groq", "gemini", "ollama", …
let client = default_for?;
let answer = client.complete?;
println!;
println!;
Examples
Each one runs against any provider you have a key for — pass its id as the first argument.
streaming is the one worth reading first if you are building a TUI: it is the whole pattern — block on a worker, drain a channel, raise a flag to stop.
Providers
| Provider | API | Credential |
|---|---|---|
| OpenAI | Responses | OPENAI_API_KEY |
| Anthropic | Messages | ANTHROPIC_API_KEY |
| OpenRouter | Chat Completions | OPENROUTER_API_KEY |
| Groq | Chat Completions | GROQ_API_KEY |
| DeepSeek | Chat Completions | DEEPSEEK_API_KEY |
| Google Gemini | OpenAI compatibility | GEMINI_API_KEY |
| xAI | Chat Completions | XAI_API_KEY |
| Mistral | Chat Completions | MISTRAL_API_KEY |
| Ollama | Local Chat Completions | none |
| ChatGPT, Claude | subscription sign-in | browser, see below |
Anything else OpenAI-shaped works without being in the table:
let client = compatible?;
Use ClientBuilder::custom for a Responses- or Anthropic-shaped endpoint. If a genuinely different wire protocol appears, implement the four-method Provider trait and nothing above it changes.
Streaming and tools
Client implements Provider. Call stream with an mpsc::Sender<Chunk> to receive text deltas, normalised tool calls, terminal usage, and errors as they arrive; complete consumes that stream into a Response. Tool schemas are translated to Responses functions, Anthropic input_schema tools, or Chat Completions functions, and results are fed back as function_call_output, tool_result blocks, or tool messages — with call ids matched, which is what every provider rejects a conversation for getting wrong.
Request::cancel takes a flag that stops a stream at the next event boundary, so a user who hits escape is not waiting on a 180-second read timeout.
The agent loop
use ;
let mut agent = new
.instructions
.max_turns;
let answer = agent.ask?;
Tools is two methods: what the tools are, and how to run one. The loop owns the rest — replaying calls and results as real conversation turns, refusing a call the model has already made verbatim, reporting a failed tool as a failure rather than as data, and forcing an answer when the turn budget runs out. Step::Planning marks the text a turn streamed before deciding to call tools, so an interface can clear a paragraph that was never the answer.
Sign-in and OAuth
API keys stay the simplest path. For a gateway that issues tokens, pass one:
let client = builder?
.bearer_token
.build?;
Implement OAuthTokenProvider for a credential that can be refreshed; Modelbridge asks for a token before every request and, on a 401 or 403, refreshes once and retries.
The oauth module implements that trait for the two consumer subscription routes, including the browser flow, the loopback redirect, the credentials file, and refresh throttling:
use ;
let store = at;
let tokens = subscription.login.run?;
let client = subscription
.client
.build?;
Both subscription routes authenticate as a vendor's own first-party CLI client against an endpoint scoped to it. That is off-label use: it can stop working without notice, and it is not what a consumer subscription is sold for. The supported path for each is the ordinary API key, and nothing above Client can tell the two apart.
Design
- No async runtime, and no dependency that pulls one in.
- Errors are a matchable
Error, not a string, so retry policy and credential refresh key off the difference between a rejected key, a rate limit, and a dead connection.Errorconverts intoStringwith?, so applications built that way lose nothing. - Rate limits and transient server faults are retried with backoff, honouring
Retry-After; rejected credentials and malformed requests are not. - All three SSE decoders tolerate unknown future event types and always terminate the stream exactly once.
- Usage is tracked globally and per
provider/model, with catalog prices producing an estimate and a provider-reported charge winning when there is one. - Vendor errors are surfaced with their human-readable message and without echoing credentials.
- Every protocol claim above is covered by a test, including real local HTTP round trips for streaming, rate-limit retry, and OAuth refresh, without contacting a vendor.
Modelbridge began as the provider layer inside a language-learning CLI, and is now the model layer for that and a terminal research agent. Adopting it deleted about 6,300 lines from the two of them.
Created by Ryan Sayer
Release history is in CHANGELOG.md. Licensed under MIT.