Skip to main content

Crate firstpass_proxy

Crate firstpass_proxy 

Source
Expand description

§firstpass-proxy

An HTTP proxy that sits as a drop-in base_url in front of Anthropic/OpenAI-compatible providers. In observe mode it forwards each request unchanged and records a tamper-evident audit trace asynchronously (zero added latency). In enforce mode it runs the escalation engine — cheapest model first, gate the output, escalate one rung on failure, serve the first output that passes — with cross-provider failover (SPEC §7.1, §7.1a, §9.1).

  • config — ProxyConfig, loaded from the environment.
  • store — the background SQLite trace writer.
  • calibrate — recalibrate the conformal serving threshold from deferred feedback.
  • upstream — BYOK passthrough to the upstream provider (observe mode).
  • provider — normalized multi-provider model access (Anthropic, OpenAI).
  • gate — runtime verification gates (Batch 3 inline set).
  • router — the enforce-mode escalation engine.
  • proxy — axum routing, observe passthrough, and enforce dispatch.
  • error — structured, no-leak error responses.
  • key_custody — per-tenant AES-256-GCM envelope key custody (ADR 0004 §D5, pre-review).
  • metrics — Prometheus recorder install + GET /metrics.
  • cli — firstpass doctor / trace logic (validate a setup, read the store).
  • mcp — minimal MCP stdio server so an agent can read its traces and submit feedback.
  • tenant_auth — experimental multi-tenant API-key auth (ADR 0004 §D1, default-off).
  • run — shared server bootstrap for the firstpass and firstpass-proxy binaries.

Re-exports§

pub use config::ProxyConfig;
pub use error::ProxyError;
pub use proxy::AppState;
pub use proxy::app;
pub use store::TraceSender;
pub use store::load_all_traces;
pub use tenant_auth::TenantId;
pub use tenant_auth::TenantKeys;

Modules§

affinity
Session promotion (SPEC §8.4): start a continuing session on the rung it already needed.
bandit
UCB1 start-rung bandit: predict-to-start, verify-to-serve.
calibrate
Recalibrate the serving threshold from real deferred feedback (SPEC §10.1, run against live traffic instead of a static benchmark suite) — the “learns your quality bar” loop.
cli
CLI surfaces for the firstpass binary (SPEC §7.3/§7.4): doctor validates a setup before you route real traffic through it, and trace reads recent audit records from the store. Both are kept here (not in the binary) so the judgment is unit-tested.
condense
Last-resort context condensing: drop the middle of a conversation that will not fit anywhere.
config
Proxy configuration, loaded from the environment (SPEC §7.4: agent-first, zero-config by default — every knob has a sane local default and can be overridden by env var).
consistency
Self-consistency uncertainty gate: measure a model’s own confidence by resampling.
decision
Decision-model gate ([[gate]] decision, SPEC §8): TypeSafe’s Jev (POST {base_url}/v1/systemone) as a cheap external verifier — ~$0.042/M input tokens versus a frontier LLM-judge call. It asks one noul (probability-of-yes) question — “does the candidate response fully and correctly satisfy the request?” — and passes iff P(yes) >= threshold.
demo
The no-keys demo: firstpass demo.
error
Structured, no-leak errors (SPEC §7.4: “errors are structured, never prose an agent must parse”).
experiment
Live monitor for certified model selection (ADR 0014).
gate
The gate framework (SPEC §8 — the moat).
gemini
Google Gemini generateContent inbound protocol — translation to/from Chat Completions.
guard
Guardrail state: outcome windows, demotion, and cooldown (ADR 0009 D3).
judge
Native LLM-judge gate (SPEC §8.3): a separate model grades the candidate output against a rubric. Three anti-gaming properties are enforced structurally here, not left to the prompt:
key_custody
Per-tenant envelope key custody (ADR 0004 §D5).
kiosk
firstpass kiosk — one command, no questions, no prior state, always ends in a receipt.
launch
firstpass launch <agent> [args…] — start a coding agent already pointed at this proxy.
mcp
Minimal MCP (Model Context Protocol) stdio server (SPEC §0.2/§7.4): lets an agent inspect and correct its own routing — read the audit traces, and submit feedback — over JSON-RPC 2.0.
metrics
Prometheus metrics: install the global recorder once per process and serve GET /metrics for scraping. Real signals only — latency, escalations, what got served, what dropped.
onboard
firstpass onboard — fully agentic onboarding (SPEC §0.2: the primary user is an agent, and the human deserves the same one-command experience). Detect the environment, plan the exact steps, execute them under --apply, and verify end-to-end — so “install firstpass and route my agent through it” is one command, not a doc to follow.
ope
Off-policy evaluation (OPE): direct-method replay and IPS/SNIPS for start-rung policies.
prior
Verified predictive routing: the I/O side. Calls a pre-generation decision-model (TypeSafe’s Jev today) with the request text and turns its answer into a decision_prior — a per-rung P(pass | rung) fed into the existing start-rung expected-cost argmin (firstpass_core::cumulative_pass, crate::bandit).
provider
Normalized model access: a provider-agnostic request/response shape, and the wire adapters (Anthropic Messages, OpenAI Chat Completions, Google Gemini) that speak it. The router (crate::router) only ever talks to Provider; it never knows which wire format is behind a given rung.
proxy
Axum wiring: routes, request/response shapes, and observe-mode trace construction (SPEC §7.1, §7.1a — forward unchanged, record asynchronously, zero added latency).
responses
OpenAI Responses API (POST /v1/responses) as a translation over the Chat Completions path.
router
The enforce-mode escalation engine (SPEC §7.1) — the crown jewel.
run
Server bootstrap shared by the firstpass and firstpass-proxy binaries: build state from config, open the trace store, and serve until Ctrl-C. Keeping this in the lib means both the unified CLI (firstpass up) and the bare proxy binary start the server the exact same way.
shadow
Shadow-spend ledger (ADR 0009 D2).
store
Tamper-evident trace storage: a background SQLite writer fed by a bounded channel.
subprocess
Subprocess gate plugin contract (SPEC §8.1) — the language-agnostic moat mechanism.
tenant_auth
Multi-tenant API-key authentication (ADR 0004 §D1) — the hosted plane’s front door.
upstream
Forwarding to the upstream Anthropic-compatible provider.
verified_cache
A response cache that can only ever return an answer which passed a gate.