Expand description
LLM reverse proxy — the core of the Gateway pillar.
Intercepts Anthropic, OpenAI, Gemini and ChatGPT traffic, compresses prompts, meters usage, and optionally translates request shapes.
§Cross-pillar coupling
When the gateway-server feature is active, start_proxy mounts
gateway_server::user_api and gateway_server::mcp::proxy routes into
the Axum router. This is intentional: the self-hosted org gateway is a
single process that combines the proxy with the admin/usage store.
The dependency is feature-gated and uni-directional at the route level
(proxy owns the router, gateway_server provides route handlers).
Modules§
- anthropic
- cache_
aligner - Cache-aligner (#940 detect, #974 relocate) — Headroom “cache aligner” port.
- cache_
attribution - Prompt-cache miss attribution (#986, cache-economics telemetry).
- cache_
breakpoint - Active prompt-cache breakpoint injection (#939, Headroom “cache aligner” adjacent).
- cache_
policy - Net-cost policy for cache-busting rewrites (#986, cache-economics).
- cache_
safety - Cache-preservation telemetry for the proxy’s frozen-region prose rewrites (#710).
- ccr
- Content-addressed recovery (CCR) for the proxy’s lossy rewrites (#482).
- chatgpt
- chatgpt_
cookies - chatgpt_
ws - WebSocket passthrough for ChatGPT’s
/backend-apirail (#597). - cold_
prefix - Big-gap cold-prefix repack prediction (#480).
- compress
- compress_
api POST /v1/compress— deterministic messages-in / messages-out compression.- cost
- Per-model proxy savings accounting.
- counterfactual
- Counterfactual savings metering (#701) — provider-authoritative receipts.
- effort
- Cache-safe, cross-provider reasoning-effort control (#834).
- forward
- gateway_
identity - Per-person gateway keys + request identity tags (enterprise#11).
- history_
prune - holdout
- Deterministic output-savings holdout (#895 Track B).
- introspect
- metrics
- models_
api GET /v1/models— the org’s model catalog, served on the proxy port (enterprise#63).- openai
- openai_
responses - openai_
responses_ ws - WebSocket bridge for the OpenAI/Codex Responses transport (#440).
- output_
savings - Output-token savings reporting (#895 Track B).
- pii
- Person pseudonymization (enterprise#39, GDPR/DSGVO).
- policy_
gate - Org-policy gateway gate (enterprise#25) — model ceiling + hard budgets,
enforced in the forward path only under a signed, trusted,
enforced = trueorg policy (crate::core::policy::org). - prose
- Frozen-region prose compression for the proxy (#710).
- prose_
ranker - Cache-safe wire prose squeeze (#895).
- providers
- Universal provider routes — the request-path half of the universal-provider-framework (enterprise#7).
- routing
- Active request router (enterprise#13) — alias + intent-tier model rewrite in the forward path, fail-open by construction.
- shape_
xlat - Cross-shape translation Anthropic ↔ OpenAI (enterprise#16, feature
shape-xlat). - tool_
kind - Classifies what produced a
tool_resultso the proxy never lossy-compresses a file/source-code read the model still needs (e.g. mid-refactor). - tool_
output - usage
- Real provider-reported token usage extraction.
- usage_
accounting - Measured-cost plumbing: request-side opt-in (#1179) and response-header extraction (#1189).
- usage_
meter - Measured per-model spend meter.
- usage_
sink - Process-wide usage event sink (enterprise#17).
- verbosity
- Cache-safe wire verbosity steer (#895 Track B).