Expand description
Active prompt-cache breakpoint injection (#939, Headroom “cache aligner” adjacent).
Anthropic’s prompt cache is opt-in per request: the client marks a
cache_control: {type:"ephemeral"} breakpoint and Anthropic caches the
prefix up to it (billing later turns at the cached rate). A raw API client
that does not set one pays full price for its (large, stable) system prompt
on every single turn. This module injects exactly one breakpoint on the
system field for those clients, so the proxy delivers the cache win the
client left on the table.
§Anthropic-only by construction
cache_control is an Anthropic concept. OpenAI Chat Completions and the
Responses API cache prefixes automatically (no per-request markers; an
injected cache_control would be ignored at best), so there is nothing to
inject there — those paths rely on OpenAI’s implicit caching and are left
byte-unchanged. This module therefore wires into the Anthropic path only.
§Safety
- Only when the client set none. The caller gates on
cached_prefix_len(messages) == 0and!prose::value_has_cache_control(system), so we never add a second breakpoint (Anthropic caps them at 4) or move a client anchor. - Exactly one, on
system— the largest, most stable prefix, fixed at the very start, so it never churns with the prune boundary. - Deterministic (#498): a pure function of the body, so the rewritten request is byte-identical across turns and the cache prefix it creates is itself stable.
- Min size. Below Anthropic’s minimum cacheable prefix the marker is ignored, so we skip tiny system prompts to avoid pointless churn.