Module prompt_cache 

Anthropic prompt-cache breakpoints (system_and_3 strategy).

Anthropic accepts up to 4 cache_control markers per request. This module places one on the system prompt (stable across the whole session) and one each on the last 3 non-system messages (rolling window of recent turns). On a multi-turn chat that means the second turn re-uses the prefix from the first, the third re-uses the second’s prefix, etc., cutting input token cost by ~75%.

Implementation note: rather than threading a cache_control field through the typed MessagesRequest/Message/ContentBlock shapes (which would pollute the OpenAI-compat path), we serialize the request to a serde_json::Value and mutate it just before send. The mutation upgrades system: "..." to system: [{type: "text", text: "...", cache_control}] and attaches cache_control to the LAST content block of each targeted message (Anthropic’s documented attachment point).

Hermes reference: agent/prompt_caching.py:apply_anthropic_cache_control.

Functions

apply_anthropic_prompt_cache

Mutate an Anthropic /v1/messages request body in-place to add prompt-cache breakpoints. Safe to call on any request: if the structure isn’t what we expect we leave it alone and the API just doesn’t cache.