Axond

Axond is a stateless, single-binary, self-hosted AI gateway. Point OpenAI or Anthropic clients at Axond to centralize provider credentials, stable model aliases, failover, usage metering, budgets, rate limits, and telemetry.

Status: beta. The supported routes and stability promises are explicit in the compatibility contract. Patch releases are upgrade-safe within 0.x; documented breaking configuration changes require a minor release.

What it provides

Axond is passthrough-first: it rewrites only model, then forwards the caller's native wire. It does not translate OpenAI payloads into Anthropic payloads or vice versa. A mismatched alias is rejected before dispatch with a typed unsupported_wire error.

Run it in two minutes

The default Compose path pulls the public release image and runs without Redis or Postgres. Placeholder provider credentials are sufficient for health, readiness, catalogue, authentication, and typed-error checks.

git clone https://github.com/Litvue/axond.git
cd axond
cp ops/compose/env.example .env
docker compose up -d

curl http://localhost:8080/healthz
curl -H 'Authorization: Bearer quickstart-platform-key' \
  http://localhost:8080/v1/models

Expected probes:

ok
{"data":[{"id":"gpt-4o",...}],"object":"list"}

To make a real provider request, replace the corresponding placeholder in .env, restart the container, and call the gateway:

curl http://localhost:8080/v1/chat/completions \
  -H 'Authorization: Bearer quickstart-platform-key' \
  -H 'content-type: application/json' \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Say hello in one word."}]}'

With placeholders, the request deliberately returns a typed provider or transport error; it still proves the complete authenticated gateway path. Keep .env until after teardown because Compose validates its required values before every command:

docker compose down -v

See Getting started for source builds, the stateful Compose profile, and SDK examples.

Install

The recommended path installs the checksum-verified prebuilt binary; it does not download the repository or invoke the Rust compiler:

# Linux x86-64 or macOS Apple Silicon
curl --proto '=https' --tlsv1.2 -LsSf \
  https://raw.githubusercontent.com/Litvue/axond/main/install.sh | sh

PowerShell on Windows x86-64:

irm https://raw.githubusercontent.com/Litvue/axond/main/install.ps1 | iex

The installers select the latest release, verify its SHA-256 sidecar, and put axond under the current user's local application directory. Download and inspect either script first if your policy forbids piped installers.

Other distribution paths:

# Cargo installs from source and compiles locally by design.
cargo install axond --locked

# Pull the current release image. There is intentionally no `latest` tag.
AXOND_VERSION=0.3.34 # x-release-please-version
docker pull "ghcr.io/litvue/axond:${AXOND_VERSION}"

Checksummed, attested prebuilt archives are published for Linux (x86_64 and aarch64, GNU and static musl each), macOS (aarch64), and Windows (x86_64). The OCI image is a multi-architecture index covering linux/amd64 and linux/arm64, so one pinned digest deploys on either architecture. Production deployments should verify the attestations and pin an image digest. See Installation and verification.

Point a client at Axond

Use http://localhost:8080/v1 as the OpenAI base URL and an Axond gateway key as the API key:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="quickstart-platform-key",
)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "hello"}],
)

Anthropic clients use the same host and their native x-api-key behavior. Detailed Python, TypeScript, streaming, Responses, embeddings, and Messages examples are in the client guides.

Choose a deployment

Axond does not terminate inbound TLS. Put it behind a trusted reverse proxy or load balancer, preserve streaming responses, and disable response buffering. Use the production checklist before exposing it outside a development network.

State tiers

State tiers describe Axond's own dependencies, not provider egress.

Configuration-owned namespaces, providers, aliases, prices, and credentials are never overridden by a datastore. See the stateful deployment guide and ADR 0017.

Tiers describe dependencies; operating modes describe ownership. Stateless remains the default, where TOML is the authority. The design for the opt-in stateful mode — durable resources in Postgres, /admin/v1 administration, and inference still served from one immutable in-memory snapshot — is ADR 0027.

A stateful replica boots and administers: it opens the control plane and serves /admin/v1 — authenticated by the [[admin_breakglass]] credential until OIDC lands — which is how durable desired state is written at all. What it cannot do yet is serve inference: a published revision does not compile into a runtime snapshot until convergence lands, so /readyz and every inference path answer 503 per request instead of the process refusing to start, and axond check preflight reports that same refusal on its serving line.

Security model

• Every route except /healthz and /readyz requires an Axond credential.
• Provider keys, inbound keys, and DSNs are referenced by environment-variable name or an explicitly supported mounted file; secret values do not belong in TOML.
• At least one static gateway key is mandatory as a breakglass path even when minted-token verification is enabled.
• Configuration and dependencies are validated before the listener binds.
• Release binaries carry checksums plus provenance and SBOM attestations; the per-architecture images add SBOM attestations, and every published manifest (children and the multi-architecture index) is signed keylessly, attested for provenance, and verified in the release workflow. Archives are not cosign-signed.

Read the deployment security model, minted-token guide, and latest security review.

To report a vulnerability, follow SECURITY.md — privately, not in an issue. It also states which releases receive fixes.

Documentation

The documentation index is organized by task:

• Getting started
• Installation and supply-chain verification
• Configuration reference
• Deployments
• Troubleshooting
• Upgrades and rollback
• Observability and runbook
• Compatibility contract
• Architecture decisions

Development

The Rust toolchain is pinned in rust-toolchain.toml. Run the core code, package, and supply-chain gate with:

just check

Useful focused checks:

just quickstart-smoke  # build current source and exercise the Compose path
just compat            # OpenAI and Anthropic SDKs against a real local axond
just tier0              # prove config-only boot in a network-denied namespace
just soak               # long streaming soak, also available on demand in CI

See CONTRIBUTING.md for contribution and compatibility-lock guidance.

Releases

Release-please maintains the changelog and workspace version. A release builds six binary targets, publishes the linux/amd64 and linux/arm64 images plus the multi-architecture index they form — booted on both architectures before it takes the <version> tag — signs and attests artifacts, and publishes gateway-core, gateway-transport, and axond to crates.io in dependency order. Maintainer procedures are in the release runbook.

License

Dual-licensed under either Apache-2.0 or MIT at your option.