Crate lean_ctx_proxy

Expand description

The LeanCTX model-API proxy, above the Engine library (lean_ctx): an edit here re-checks this crate and the binary, not the Engine.

LLM reverse proxy — the core of the Gateway pillar.

Intercepts Anthropic, OpenAI, Gemini and ChatGPT traffic, compresses prompts, meters usage, and optionally translates request shapes.

§TLS Requirements

The proxy does NOT terminate TLS. For production/multi-tenant deployments, place behind a TLS-terminating reverse proxy (nginx, caddy, traefik). For mTLS, configure the reverse proxy with client certificates. See: https://github.com/yvgude/lean-ctx/issues/1371

§Org gateway coupling (ADR-023)

The self-hosted org gateway (lean-ctx-enterprise) extends this proxy with admin/usage-store routes. In the OSS build the proxy runs standalone.

Modules§

auto_routing: Legacy model=auto wire types, kept for compatibility only.
break_even
cache_aligner: Cache-aligner (#940 detect, #974 relocate) — Headroom “cache aligner” port.
cache_attribution: Prompt-cache miss attribution (#986, cache-economics telemetry).
cache_policy: Net-cost policy for cache-busting rewrites (#986, cache-economics).
cache_safety: Cache-preservation telemetry for the proxy’s frozen-region prose rewrites (#710).
cache_share: Cache-share guard: per provider+model cost log, rolling cache-read share, alarm below the threshold, and the provider allowlist for open models.
ccr: In-band CCR on the proxy wire (#493): <lc_expand:HASH> markers a model echoed are spliced back into the request body before it leaves.
circuit_breaker: Public deterministic circuit-breaker reference.
cold_prefix: Big-gap cold-prefix repack prediction (#480).
compress
compress_api: POST /v1/compress — deterministic messages-in / messages-out compression.
compress_shared: Provider-neutral classification for content blocks compressed in place.
compression_savings: Input-compression savings from the [proxy] compression_holdout (#1905).
cost: Per-model proxy savings accounting.
counterfactual: Counterfactual savings metering (#701) — provider-authoritative receipts.
dedup: Request-local, lossless deduplication of repeated tool output.
determinism_guard: Formal, verifiable prompt-cache safety for proxy request rewrites.
effort: Cache-safe, cross-provider reasoning-effort control (#834).
eligibility: Static eligibility evaluation for the public proxy boundary.
history_prune
holdout: Deterministic output-savings holdout (#895 Track B).
introspect
live_zone: Live-zone-aware request compression preserves provider-cached prefixes.
output_savings: Output-token savings reporting (#895 Track B).
pii: Person pseudonymization (enterprise#39, GDPR/DSGVO).
pipeline_bench: End-to-end, deterministic workloads for the proxy compression pipeline.
pre_optimize
prefix_cache_stats: Structured prefix-cache statistics for the /status endpoint.
prefix_replay: Append-only delta detection and forwarded-byte replay for provider prefix cache stability.
prose: Frozen-region prose compression for the proxy (#710).
prose_compress: Markdown-aware normalization and task-relevance filtering.
prose_patterns: Deterministic, loss-conscious compression for repetitive prose.
prose_ranker: Cache-safe wire prose squeeze (#895).
rollout: Public rollout cohort schema and deterministic assignment reference.
routing: Explicit model targets — operator-written aliases in the forward path.
session_elide: Long-session elision for Anthropic Messages requests.
sticky_tools: Sticky CCR tool injection for provider prefix-cache stability.
tool_kind: Classifies what produced a tool_result so the proxy never lossy-compresses a file/source-code read the model still needs (e.g. mid-refactor).
usage: Real provider-reported token usage extraction.
usage_meter: Measured per-model spend meter.
usage_parity: Proxy-to-kernel provider and usage conversion.
usage_sink: Process-wide usage event sink (enterprise#17).
verbosity: Cache-safe wire verbosity steer (#895 Track B).

Functions§

register_shapers: Hand the proxy’s output shaping to the engine’s ctx_shape tool. The CLI calls this at start-up; the first registration wins.