Module cache_keepwarm 

Prompt-cache keep-warm policy (task #444), owned by the session actor.

Why the actor: it is the one place that knows a session is idle (turn finished, nothing queued, no compaction, no prompt, no driver), when it parks or ends, and what it has spent. The ping itself is Runtime::cache_keepwarm_ping (runtime::keepwarm).

Policy (docs/specs/cache-ttl-spec.md §7)

An idle period starts when a turn whose last Anthropic request was accepted AND cached something (cache_read + cache_write > 0) ends with a finished assistant reply, cache_keepwarm is not off, and the next turn would not roll the context over first. It holds:

• the anchor: send time of the last accepted request (monotonic + wall clock). The provider’s TTL runs from the start of that request.
• the honoured TTL: what the server actually keeps, not what is configured. 1h only with evidence (a 1h write in the last request or earlier in the session, no silent-downgrade latch); otherwise 5m. hybrid is governed by its 5m history marker. Pings are due at anchor + TTL − margin (55 min / 4.5 min), on whichever of the monotonic and wall clocks comes first (the monotonic clock stops while the machine sleeps; the provider’s TTL does not).
• the prefix fingerprint recorded at idle. Every ping re-builds the prefix and is refused (period ends) if it no longer matches.
• a ski-rental ledger of measured spend. A ping is only sent while ledger + worst-case ping ≤ bound, where the bound is re-writing the prefix at the honoured TTL’s write price, and the worst case is a full read of the prefix + the known write (ping 1: the last turn’s uncached tail and reply) + uncached input + max_tokens of output. Under budgeted thinking max_tokens is budget + 1, so such pings only run for prefixes large enough that one worst-case ping fits the bound.
• the cap exposure: max_session_cost is a hard cap, so admission prices what a dispatched ping can actually bill — the larger of that read-path worst case and a full miss (the whole prefix written at the 1h price if the request carries 1h markers, else the honoured TTL’s, plus the same uncached input and max_tokens of output). The ski-rental bound stays on the read path: a miss is the rewrite and ends the period when it happens.

The period ends on the first of: bound, <N>h window, prefix change, context rollover ahead, a ping that cached nothing / missed / rewrote a large share of the prefix / paid for a large uncached share, expiry, a 4xx, exhausted transient retries, max_session_cost, a turn or compaction starting, an explicit park, /keepwarm off, or the session ending. While a period is active the session does not auto-park.

Structs

EpochCarry

What a cache epoch carries from one period to the next. An epoch is the life of one real request’s cache entry: only a new real request (new usage, new anchor) starts a fresh one. Ending a period (e.g. /keepwarm off) and starting another without a real turn in between CONTINUES the epoch: same ledger, same idle start, ping 1’s write allowance not re-granted.

Period

One idle period. Pure: every clock reading is passed in.

PeriodConfig

What the period needs to know about the session at start.

TtlEvidence

Evidence about the TTL the server honours for this session.

Enums

Fire

Timer decision.

PeriodEnd

Why a period ended (or never started). Logged once, shown by /keepwarm status.

Constants

BACKOFF

Delays before retrying a transient ping failure; then the period ends.

LATE_GUARD

No ping may start once the cache is this close to expiry: the request still has to reach the provider (connect, upload a large body).

PING_INPUT_ESTIMATE

Estimated uncached input of a ping before one has been measured: the trailing ping message plus framing.

SHARE_LIMIT_PERCENT

A ping whose uncached input, or (after ping 1) whose cache write, exceeds this share of the prefix ends the period: it is paying for uncached tokens or rewriting history, not refreshing a cache. 10%: the trailing ping message is ~40 tokens, under 4% of the smallest cacheable prefix (1,024 tokens), so a healthy ping never comes close.

WALL_POLL

While a period is active the actor re-reads the wall clock at least this often: after a suspend the monotonic deadline is late by the time slept.

Functions

governing_ttl

TTL of the entry that covers the history prefix: the shortest TTL among the breakpoints a ping must keep alive. Anthropic requires longer TTLs to precede shorter ones, so under hybrid the history (tail) marker is the 5m one.

honoured_ttl

The TTL the server actually honours for the history entry. 1h needs positive evidence; with none (or a downgrade) a 1h configuration is scheduled — and its rewrite priced — as 5m.

interval_override

Test seam replacing ping_interval(honoured TTL).

ping_interval

Ping interval for ttl, measured from the anchor: TTL − margin.

request_cost_usd

Cost of one request: read cache reads, write cache writes at ttl’s write price, input uncached input, output output tokens.

rewrite_cost_usd

Cost of re-writing prefix_tokens at the TTL’s write price: what the next turn pays if the cache lapses — the ski-rental “buy” price.

safety_margin

Safety margin before the TTL: 5 min for 1h, 30 s for 5m.

Type Aliases

OverrideSlot

The per-session /keepwarm override, readable without the actor.