Module cache_policy 

Net-cost policy for cache-busting rewrites (#986, cache-economics).

The cold-prefix repack (#480) re-seeds a leaner prompt cache when the proxy predicts the client-cached prefix has already gone cold (idle past the TTL). Because the entry is already expired, the provider re-writes the prefix on the next turn no matter what — so compressing that unavoidable re-write is free savings, except for prefixes too small to be cached at all. Repacking one of those only churns the conversation’s cache identity for no benefit.

This module is the pricing brain that decides when a repack pays:

• worth_repacking — the live gate applied in anthropic.rs. It runs before compression, so it can only weigh the measurable precondition: is the cacheable prefix even large enough to be worth re-seeding?
• net_cost_decision / repack_saving_usd — the fully priced primitive (before/after token counts × ModelCost) for callers that already know the compressed size (tests today, cache-edit batching later).

Pure functions, no globals; gated behind the opt-in proxy.cache_policy at the call site so a default proxy keeps today’s behaviour exactly.

Constants

MIN_CACHEABLE_TOKENS

Minimum cacheable prefix size, in tokens. Anthropic will not cache a prefix below this (1024 for most models; Haiku needs more), so re-seeding a smaller one can never produce a cache the provider would keep — the conservative floor below which a repack is pure churn. Chosen as the documented Anthropic minimum rather than an estimate.

Functions

net_cost_decision

Fully priced repack decision for callers that already know the compressed size. True when the prefix is cacheable and re-seeding it strictly lowers the unavoidable cold re-write cost. The precondition mirrors worth_repacking so the live gate and the priced primitive never disagree.

prefix_tokens

Token count of the prefix the provider would actually cache: the system field plus the client-cached messages messages[0..cached]. Measured (not estimated) via the same BPE counter the rest of the proxy uses, so the gate reflects the real prefix — including the system prose a cold-prefix repack re-seeds, which is usually the bulk of it.

repack_saving_usd

Cache-write cost saved by re-seeding a compressed prefix instead of the full one, in USD. On a cold prefix the provider re-writes regardless, so the saving is the avoided write of the dropped tokens: (before − after) × cache_write. Clamped to 0.0 when compression did not shrink the prefix.

worth_repacking

Live repack gate (pre-compression). A cold-prefix repack only pays when the cacheable prefix (system + cached messages) is large enough that the provider will actually cache the re-seeded version; below MIN_CACHEABLE_TOKENS the repack just churns the conversation’s cache key. Applied as an extra AND-condition on the existing repack decision, so the policy can only make repacking more conservative — never trigger a rewrite that would not have happened.