Expand description
LLM reverse proxy — the core of the Gateway pillar.
Intercepts Anthropic, OpenAI, Gemini and ChatGPT traffic, compresses prompts, meters usage, and optionally translates request shapes.
§TLS Requirements
The proxy does NOT terminate TLS. For production/multi-tenant deployments, place behind a TLS-terminating reverse proxy (nginx, caddy, traefik). For mTLS, configure the reverse proxy with client certificates. See: https://github.com/yvgude/lean-ctx/issues/1371
§Org gateway coupling (ADR-023)
The self-hosted org gateway (lean-ctx-enterprise) extends this proxy with admin/usage-store routes. In the OSS build the proxy runs standalone.
Modules§
- anthropic
- bedrock
- Amazon Bedrock Runtime request validation and AWS Signature Version 4.
- break_
even - cache_
aligner - Cache-aligner (#940 detect, #974 relocate) — Headroom “cache aligner” port.
- cache_
attribution - Prompt-cache miss attribution (#986, cache-economics telemetry).
- cache_
breakpoint - Active prompt-cache breakpoint injection (#939, Headroom “cache aligner” adjacent).
- cache_
policy - Net-cost policy for cache-busting rewrites (#986, cache-economics).
- cache_
safety - Cache-preservation telemetry for the proxy’s frozen-region prose rewrites (#710).
- ccr
- Content-addressed recovery (CCR) for the proxy’s lossy rewrites (#482).
- chatgpt
- chatgpt_
cookies - chatgpt_
ws - WebSocket passthrough for ChatGPT’s
/backend-apirail (#597). - cold_
prefix - Big-gap cold-prefix repack prediction (#480).
- compress
- compress_
api POST /v1/compress— deterministic messages-in / messages-out compression.- compress_
shared - Provider-neutral classification for content blocks compressed in place.
- cost
- Per-model proxy savings accounting.
- counterfactual
- Counterfactual savings metering (#701) — provider-authoritative receipts.
- dedup
- Session-scoped cache for deduplicating compressed tool results.
- effort
- Cache-safe, cross-provider reasoning-effort control (#834).
- effort_
routing - Per-turn effort routing (#1148, opt-in dynamic thinking budget).
- forward
- Shared upstream forward path for OpenAI-compatible providers.
- gateway_
identity - Per-person gateway keys + request identity tags (enterprise#11).
- history_
prune - holdout
- Deterministic output-savings holdout (#895 Track B).
- image_
compression - Image compression for vision-model content (#1149).
- introspect
- latency_
guard - metrics
- model_
router - Intent-based model routing (P8 / DIM 3 — Leistungsstufe).
- models_
api GET /v1/models— the org’s model catalog, served on the proxy port (enterprise#63).- ocla_
cache_ bridge - Adapter between proxy request fields and the OCLA response cache.
- openai
- openai_
responses - openai_
responses_ ws - WebSocket bridge for the OpenAI/Codex Responses transport (#440).
- output_
savings - Output-token savings reporting (#895 Track B).
- pii
- Person pseudonymization (enterprise#39, GDPR/DSGVO).
- policy_
gate - Org-policy gateway gate (enterprise#25) — model ceiling + hard budgets,
enforced in the forward path only under a signed, trusted,
enforced = trueorg policy (crate::core::policy::org). - prefix_
cache_ stats - Structured prefix-cache statistics for the
/statusendpoint. - prefix_
replay - Append-only delta detection and forwarded-byte replay for provider prefix cache stability.
- prose
- Frozen-region prose compression for the proxy (#710).
- prose_
ranker - Cache-safe wire prose squeeze (#895).
- providers
- Universal provider routes — the request-path half of the universal-provider-framework (enterprise#7).
- quality_
lab_ api - POST /v1/quality-lab — run Quality Lab analysis on a text pair.
- response_
optimizer - Response Optimizer (P9 / DIM 2 — Output-Optimierung).
- routing
- Active request router (enterprise#13) — alias + intent-tier model rewrite in the forward path, fail-open by construction.
- routing_
feedback - Adapter between proxy routing events and OCLA quality tracking.
- shape_
xlat - Cross-shape translation Anthropic ↔ OpenAI (enterprise#16, feature
shape-xlat). - sse_
keepalive - SSE keepalive injection for proxy-to-client streams.
- sticky_
tools - Sticky CCR tool injection for provider prefix-cache stability.
- tool_
kind - Classifies what produced a
tool_resultso the proxy never lossy-compresses a file/source-code read the model still needs (e.g. mid-refactor). - tool_
output - usage
- Real provider-reported token usage extraction.
- usage_
accounting - Measured-cost plumbing: request-side opt-in (#1179) and response-header extraction (#1189).
- usage_
azure - Azure-specific response detection and usage normalization.
- usage_
meter - Measured per-model spend meter.
- usage_
parity - Proxy-to-kernel provider and usage conversion.
- usage_
sink - Process-wide usage event sink (enterprise#17).
- verbosity
- Cache-safe wire verbosity steer (#895 Track B).
- web_app