nrouter 3.1.5

nRouter SDK — one API key for models across six provider clouds
Documentation

nRouter SDK for Rust

One API key for models across six provider clouds.

TypeSafe Jev System One

demo/jev_system_one.rs sends a standard chat-completions request to typesafe/jev and prints the decision plus normal nRouter metadata. See the cross-SDK JEV Examples index for prerequisites and feature limits.

Installation

Add nrouter to your Cargo.toml:

[dependencies]
nrouter = "3.1.5"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Or for development from a local repository checkout:

[dependencies]
nrouter = { path = "../nrouter-sdk/sdks/rust" }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Authentication & Setup

Set your API key in the environment:

export NROUTER_API_KEY="sk-nrouter-your-api-key-here"

Two entry points, on purpose

nrouter::client() returns an [async-openai] client pointed at the gateway, so every OpenAI-shaped call you already write keeps working:

use async_openai::types::chat::{
    ChatCompletionRequestUserMessageArgs, CreateChatCompletionRequestArgs,
};

let client = nrouter::client()?;               // reads NROUTER_API_KEY
let request = CreateChatCompletionRequestArgs::default()
    .model("gpt-5.4-mini")
    .messages(vec![ChatCompletionRequestUserMessageArgs::default()
        .content("Hello!").build()?.into()])
    .build()?;
let response = client.chat().create(request).await?;

nrouter::http::Client is a native client that hands back the x-nr-* metadata. async-openai discards the raw response, so per-request cost, token counts and cache outcome are unreachable through it — that is the whole reason this second path exists:

Custom gateway URLs must use HTTPS. Plain HTTP is accepted only for localhost or a loopback IP so local development remains possible without ever sending an API key unencrypted to a remote host.

use serde_json::json;

let client = nrouter::http::Client::from_env()?;
let out = client.chat_completions(&json!({
    "model": "gpt-5.4-mini",
    "messages": [{"role": "user", "content": "Hello!"}]
})).await?;

// Branch on the status, never on `cost` being None-ish. An unpriced model
// reports cost == None, and rendering that as 0.0 reports a free request —
// which no enabled model is.
match out.meta.cost {
    Some(usd) => println!("cost ${usd}"),
    None => println!("unpriced ({:?})", out.meta.cost_status),
}

ResponseMeta carries every x-nr-* header the gateway emits, named in snake_case — x-nr-request-cost becomes cost and x-nr-cache-read-tokens becomes cache_read_tokens. The authoritative set is spec/gateway-response-headers.json, derived from the gateway and held against this SDK by conformance/check_conformance.py. This page does not restate it: a copied list of a set that grows is a list that goes stale, and under the word "every" it becomes a false claim of exhaustiveness rather than a stale number.

Streaming

The native client incrementally parses SSE on all four text wires. delta is portable across OpenAI and Anthropic frames; raw preserves the complete provider-native JSON. Dropping the stream drops the response body, cancelling unread generation:

let mut stream = client.messages_stream(&serde_json::json!({
    "model": "claude-haiku-4-5-20251001",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Hello!"}]
})).await?;

while let Some(chunk) = stream.next().await? {
    print!("{}", chunk.delta);
}
println!("\nrequest {:?}", stream.meta.request_id);

chat_completions_stream, completions_stream, messages_stream, and responses_stream clone the request body and force stream: true. An in-band output-guardrail event returns NRouterError::GuardrailBlocked; a bare EOF is a retryable transport failure, never a clean completion.

Errors

NRouterError variants are chosen from the gateway's stable code — not the HTTP status alone, which cannot separate multiple 400, 402 or 429 conditions.

match client.chat_completions(&body).await {
    Err(NRouterError::GuardrailBlocked(b)) => { /* change the request */ }
    Err(NRouterError::Credit(b))           => { /* top up */ }
    Err(e) if e.is_retryable()             => { /* rate limit, service, transport */ }
    Err(e) => return Err(e.into()),
    Ok(out) => { /* ... */ }
}
Variant Code(s) HTTP
Request invalid_request, input_too_large, max_output_tokens_too_large, fallback_not_allowed, guardrail_not_found, image_input_unsupported, pot_route_feature_unsupported, cost_band_unsatisfiable 400
GuardrailBlocked guardrail_blocked 400
Authentication invalid_api_key, byok_credential_rejected 401
Credit insufficient_credits, plan_allowance_exhausted, plan_required, insufficient_promo_credit, insufficient_hack_credit, byok_plan_required 402
NotFound model_not_found 404
RateLimit rate_limit_exceeded, tpm_limit_exceeded, pot_spend_limit_exceeded, byok_fair_use_exceeded 429
Service credit_check_failed, service_unavailable 503
Other model_denied_by_org, model_denied_by_team, model_denied_by_key, byok_credential_unavailable, anything newer than this SDK 403 / 409 / —
Transport never reached the gateway —

is_retryable() is true only for RateLimit, Service and Transport. Every other variant names something permanent, where a retry burns quota and cannot change the answer.

ErrorBody::limit_source names WHICH ceiling refused a 429. It is None when the gateway could not attribute the refusal, and this SDK does not guess — sending a customer to raise the wrong limit is worse than saying nothing.

Configuration

let client = nrouter::http::Client::new("sk-nrouter-...")?
    .with_base_url("https://api-stage.nrouter.ai/v1")
    .with_http_client(my_reqwest_client);      // proxy, timeouts, pooling

Endpoints

All 15 gateway operations have named buffered helpers: chat_completions, completions, embeddings, images_generations, messages, count_tokens, responses, models, model, create_video, retrieve_video, download_video_content, audio_speech, audio_transcriptions, and audio_translations. post, get, bytes, multipart, and stream remain available as escape hatches.

Not JSON: audio_transcriptions and audio_translations send multipart/form-data (the gateway requires a binary file part, so the JSON helpers cannot reach them); audio_speech() and download_video_content() return raw bytes plus metadata; bytes(method, path, body) remains available for other non-JSON responses. The JSON helpers refuse a non-JSON response rather than handing back an empty body for a request you were billed for.

Build and test

cargo test                                   # unit + integration + doc tests
cargo clippy --all-targets -- -D warnings

MSRV 1.88 — the effective floor of the dependency graph (icu_* via reqwest's TLS stack declare 1.88). Derive it rather than trusting this line:

cargo metadata --format-version 1 --locked \
  | python3 -c "import json,sys;print(max((p['rust_version'] for p in json.load(sys.stdin)['packages'] if p.get('rust_version')), key=lambda v:[int(x) for x in v.split('.')]))"
``` Publishing: [PUBLISHING.md](PUBLISHING.md).

[`async-openai`]: https://crates.io/crates/async-openai

## How guardrails, budgets and routing work

They are configured in the dashboard and enforced at the **gateway**, not in
this package. The useful guarantee is not that they are always on — it is that
**whatever you have enabled cannot be bypassed by a client**, this one
included, and behaves identically from every nRouter SDK and from raw `curl`.

- [Guardrails](https://nrouter.ai/docs/guides/guardrails) — PII redaction,
  injection protection, secret and keyword scanning, pre-call and post-call.
  Which ones run is resolved per request: the organization's guardrail switch
  first, then the narrowest applicable assignment wins across
  key > team > org > default, and a winner disabled at that scope does not run.
- [Budget controls](https://nrouter.ai/docs/guides/budget-controls) — spend
  limits per key, team and organization.
- [Observability](https://nrouter.ai/docs/guides/observability) — cost and usage
  on billable calls. Free routes are genuinely free and carry no
  `x-nr-request-cost`: `/v1/messages/count_tokens`, and video polling and
  content retrieval.

[Routing and fallbacks](https://nrouter.ai/docs/guides/router-settings)
carry two conditions worth knowing before you rely on failover you have not
enabled:

- **Two ways to ask for a model, chosen by what you put in `model`.**
  *Direct*: name the model; nRouter never swaps it for another. A concrete
  model is never re-routed and inherits no hidden fallback — a different model
  answers only through a fallback you configured yourself (Router Settings, or
  `nrouter_fallbacks` on the request). *`nrouter/auto`*: nRouter chooses an
  eligible model for you. Either way you pay the served model's list price,
  and `x-nr-model` names it.
- **Selection strategies on `nrouter/auto`.** `FeatureOptions.optimize` sets
  `nrouter_optimize` to `"balanced"` (org default across all price bands with
  no ceiling), `"quality"` (small cost weight across all price bands), or
  `"cost"` (restricted to the `cost_band` ceiling). For cost mode, set `FeatureOptions { optimize:
  Some("cost".into()), cost_band: Some("low".into()), ..Default::default() }`
  (`"mid"` is the default band, `"high"` admits every band) and merge
  `nrouter::options::build_extra_body(&opts)?` into the request body; it maps onto
  `nrouter_optimize` and `nrouter_cost_band` and refuses any other value
  locally. The band caps each model's blended list rate
  `(3 * input + 1 * output) / 4` per 1M tokens: `low` under $1, `mid` under $7
  (low included), `high` every band. Sending `nrouter_cost_band` on a Direct
  model or under `"quality"` or `"balanced"` is refused with `400 invalid_request`;
  a band with no eligible model is refused with `400 cost_band_unsatisfiable`
  before any spend.
- **Text wires only** — chat completions, responses, messages and legacy
  completions. Audio, image and video calls take a single-provider route and
  have no cross-provider failover.
- [API reference](https://nrouter.ai/docs/api-reference) — the wire
  contract every SDK here implements.

## Demos & Examples

Runnable demonstrations live in [`demo/`](demo/):
- [Quickstart Demo](demo/quickstart.rs) — demonstrates Rust client initialization, streaming, and metadata inspection.
- [Demo Documentation](demo/README.md) — cargo run instructions.

## Validation Playbook

This SDK maintains a repeatable 18-step verification process:
- [Validation Playbook](docs/validation-playbook.md) — comprehensive end-to-end verification runbook.

## Open-Source Standards & License

- **License:** [MIT License](../../LICENSE)
- **Repository:** [nRouterGateway/nrouter-sdk](https://github.com/nRouterGateway/nrouter-sdk)
- **Issue Tracker:** [GitHub Issues](https://github.com/nRouterGateway/nrouter-sdk/issues)