Module cache_policy 

Net-cost policy for cache-busting rewrites (#986, cache-economics).

The cold-prefix repack (#480) re-seeds a leaner prompt cache when the proxy predicts the client-cached prefix has already gone cold (idle past the TTL). Because the entry is already expired, the provider re-writes the prefix on the next turn no matter what — so compressing that unavoidable re-write is free savings, except for prefixes too small to be cached at all. Repacking one of those only churns the conversation’s cache identity for no benefit.

This module is the pricing brain that decides when a repack pays:

• worth_repacking — the live gate applied in anthropic.rs. It runs before compression, so it can only weigh the measurable precondition: is the cacheable prefix even large enough to be worth re-seeding?
• net_cost_decision / repack_saving_usd — the fully priced primitive (before/after token counts × ModelCost) for callers that already know the compressed size (tests today, cache-edit batching later).
• system_mutation_decision — the warm-prefix gate for compressing the system prompt while a message prefix is cached (#1912): it prices the bust of the whole prefix against the per-turn cache-read saving, with the conversation’s own turn count as the reuse estimate and current prices from the shared table (model_cost_for).

Pure functions, no globals; gated behind the opt-in proxy.cache_policy at the call site so a default proxy keeps today’s behaviour exactly.

Enums

MutationDecision

Decision outcome for a frozen-region mutation.

Constants

MIN_CACHEABLE_TOKENS

Minimum cacheable prefix size, in tokens. Anthropic will not cache a prefix below this (1024 for most models; Haiku needs more), so re-seeding a smaller one can never produce a cache the provider would keep — the conservative floor below which a repack is pure churn. Chosen as the documented Anthropic minimum rather than an estimate.

Functions

model_cost_for

Cost parameters for a model name, from the shared pricing table (embedded prices, [cost.prices] overrides, live list) so the gate never prices a request with stale per-family constants (#1912).

net_cost_decision

Fully priced repack decision for callers that already know the compressed size. True when the prefix is cacheable and re-seeding it strictly lowers the unavoidable cold re-write cost. The precondition mirrors worth_repacking so the live gate and the priced primitive never disagree.

observed_prefix_reuse

Expected future reuses of a cached prefix, read from the conversation itself: every prior assistant turn is a request that already re-sent the prefix. A pure function of the request, so the gate below decides the same way on every replay of the same turn (#498) — unlike a process-wide rate.

prefix_tokens

Token count of the prefix the provider would actually cache: the system field plus the client-cached messages messages[0..cached]. Measured (not estimated) via the same BPE counter the rest of the proxy uses, so the gate reflects the real prefix — including the system prose a cold-prefix repack re-seeds, which is usually the bulk of it.

repack_saving_usd

Cache-write cost saved by re-seeding a compressed prefix instead of the full one, in USD. On a cold prefix the provider re-writes regardless, so the saving is the avoided write of the dropped tokens: (before − after) × cache_write. Clamped to 0.0 when compression did not shrink the prefix.

should_mutate_frozen

Generalised net-cost gate for any frozen-region mutation. A frozen region is served from the prompt cache, so:

system_mutation_decision

Priced gate for compressing the system prompt while the client caches a message prefix (#1912). The system block sits in front of every cached message, so rewriting it busts the whole prefix: the bust is priced on prefix_before (system + cached messages, see prefix_tokens) and the saving on the measured shrink of the system block alone.

worth_repacking

Live repack gate (pre-compression). A cold-prefix repack only pays when the cacheable prefix (system + cached messages) is large enough that the provider will actually cache the re-seeded version; below MIN_CACHEABLE_TOKENS the repack just churns the conversation’s cache key. Applied as an extra AND-condition on the existing repack decision, so the policy can only make repacking more conservative — never trigger a rewrite that would not have happened.