modelbridge 0.5.2

One provider-neutral Rust client for OpenAI, Anthropic, Gemini, Groq, Ollama and friends: streaming, tool calling, an agent loop, usage accounting, and subscription sign-in
Documentation

modelbridge

Modelbridge is one small Rust API for the language-model providers people actually use. Pick a provider and a model; the crate already knows the endpoint, the credential variable, the request shape, the streaming protocol, the tool-call format, and the token prices. It is synchronous and runtime-free — no Tokio, no async colouring — so a CLI, TUI, or desktop app can run a stream on a worker thread and drain a channel each frame. On top of the transport it ships the two things applications otherwise rewrite badly: a tool-calling agent loop, and browser sign-in for the ChatGPT and Claude subscription routes.

Layout

  • src/catalog.rs — built-in providers and a curated, priced model list. Never an allow-list: any model id works, including one released tomorrow.
  • src/client.rs — the HTTP client: one route, one credential, retries, and a usage meter.
  • src/encode.rs — one request translated into Responses, Anthropic Messages, or Chat Completions, tool round trips included.
  • src/stream.rs — the three server-sent-event dialects decoded into Chunks.
  • src/agent.rs — the call-tools-until-answered loop.
  • src/oauth/ — PKCE, the loopback redirect catcher, token storage, and refresh for the subscription routes.
  • src/meter.rs — token and cost accounting, in total and by provider/model.

Install

[dependencies]

modelbridge = "0.4"

Skip the subscription sign-in machinery, and its three dependencies, with default-features = false.

Quickstart

use modelbridge::{Client, Request};

// Reads ANTHROPIC_API_KEY. Try "openai", "groq", "gemini", "ollama", …
let client = Client::default_for("anthropic")?;

let answer = client.complete(
    &Request::new("Be concise and practical.")
        .user("Why is the sky blue?")
        .max_output_tokens(300),
)?;

println!("{}", answer.text);
println!("{} tokens, about ${:.4}", answer.usage.total(), answer.usage.dollars());

Examples

Each one runs against any provider you have a key for — pass its id as the first argument.

cargo run --example basic     -- groq        # one completion, with cost

cargo run --example streaming -- anthropic   # worker thread, channel, cancellation

cargo run --example tools     -- openai      # a tool round trip by hand

cargo run --example agent     -- anthropic   # the same thing, driven by the agent loop

cargo run --example signin    -- claude      # subscription sign-in, start to finish

streaming is the one worth reading first if you are building a TUI: it is the whole pattern — block on a worker, drain a channel, raise a flag to stop.

Providers

Provider API Credential
OpenAI Responses OPENAI_API_KEY
Anthropic Messages ANTHROPIC_API_KEY
OpenRouter Chat Completions OPENROUTER_API_KEY
Groq Chat Completions GROQ_API_KEY
DeepSeek Chat Completions DEEPSEEK_API_KEY
Google Gemini OpenAI compatibility GEMINI_API_KEY
xAI Chat Completions XAI_API_KEY
Mistral Chat Completions MISTRAL_API_KEY
Ollama Local Chat Completions none
ChatGPT, Claude subscription sign-in browser, see below

Anything else OpenAI-shaped works without being in the table:

let client = modelbridge::Client::compatible(
    "https://models.internal.example/v1/chat/completions",
    "company-model-v2",
    std::env::var("COMPANY_MODEL_KEY").unwrap(),
)?;

Use ClientBuilder::custom for a Responses- or Anthropic-shaped endpoint. If a genuinely different wire protocol appears, implement the four-method Provider trait and nothing above it changes.

Streaming and tools

Client implements Provider. Call stream with an mpsc::Sender<Chunk> to receive text deltas, normalised tool calls, terminal usage, and errors as they arrive; complete consumes that stream into a Response. Tool schemas are translated to Responses functions, Anthropic input_schema tools, or Chat Completions functions, and results are fed back as function_call_output, tool_result blocks, or tool messages — with call ids matched, which is what every provider rejects a conversation for getting wrong.

Request::cancel takes a flag that stops a stream at the next event boundary, so a user who hits escape is not waiting on a 180-second read timeout.

The agent loop

use modelbridge::agent::{Agent, Step, Tools};

let mut agent = Agent::new(&client, my_tools)
    .instructions("Search before answering anything time-sensitive.")
    .max_turns(6);

let answer = agent.ask("What shipped in Rust 1.90?", &mut |step| {
    if let Step::Token(text) = step { print!("{text}") }
})?;

Tools is two methods: what the tools are, and how to run one. The loop owns the rest — replaying calls and results as real conversation turns, refusing a call the model has already made verbatim, reporting a failed tool as a failure rather than as data, and forcing an answer when the turn budget runs out. Step::Planning marks the text a turn streamed before deciding to call tools, so an interface can clear a paragraph that was never the answer.

Sign-in and OAuth

API keys stay the simplest path. For a gateway that issues tokens, pass one:

let client = modelbridge::Client::builder("openai", "gpt-5.6-terra")?
    .bearer_token(access_token)
    .build()?;

Implement OAuthTokenProvider for a credential that can be refreshed; Modelbridge asks for a token before every request and, on a 401 or 403, refreshes once and retries.

The oauth module implements that trait for the two consumer subscription routes, including the browser flow, the loopback redirect, the credentials file, and refresh throttling:

use modelbridge::oauth::{Store, Subscription};

let store = Store::at(config_dir.join("credentials.json"));
let tokens = subscription.login("My App").run(&store, &announce, &cancel)?;
let client = subscription
    .client(subscription.session(&store, tokens), "claude-sonnet-5")
    .build()?;

Both subscription routes authenticate as a vendor's own first-party CLI client against an endpoint scoped to it. That is off-label use: it can stop working without notice, and it is not what a consumer subscription is sold for. The supported path for each is the ordinary API key, and nothing above Client can tell the two apart.

Design

  • No async runtime, and no dependency that pulls one in.
  • Errors are a matchable Error, not a string, so retry policy and credential refresh key off the difference between a rejected key, a rate limit, and a dead connection. Error converts into String with ?, so applications built that way lose nothing.
  • Rate limits and transient server faults are retried with backoff, honouring Retry-After; rejected credentials and malformed requests are not.
  • All three SSE decoders tolerate unknown future event types and always terminate the stream exactly once.
  • Usage is tracked globally and per provider/model, with catalog prices producing an estimate and a provider-reported charge winning when there is one.
  • Vendor errors are surfaced with their human-readable message and without echoing credentials.
  • Every protocol claim above is covered by a test, including real local HTTP round trips for streaming, rate-limit retry, and OAuth refresh, without contacting a vendor.

Modelbridge began as the provider layer inside a language-learning CLI, and is now the model layer for that and a terminal research agent. Adopting it deleted about 6,300 lines from the two of them.

Created by Ryan Sayer

Release history is in CHANGELOG.md. Licensed under MIT.