Module tool_keepwarm 

Keep the prompt cache warm while a tool round runs (cache_keepwarm_tools).

A provider round that asks for tools is followed, once they return, by a request that re-sends that round’s whole prefix. If the tools take longer than the cache TTL (workers are pinned to 5m and routinely run builds, test sweeps and sleep; poll loops for that long), the entry has expired by then and the next request rewrites the whole prefix at the write price (1.25× on 5m) instead of reading it (0.1×, 0.05× on the 5.5 models).

While the round’s tools run, this task re-sends that round’s own request — the exact request messages, tools, system prompt, thinking config and options it went out with — plus one trailing ping text block (KeepWarmMode::Replay, appended by the transport after marker placement), shortly before the TTL lapses, and again after every accepted ping, until the tools return, the turn is cancelled or the stream ends (the guard drops). It is a keep-warm ping in every other respect (runtime::keepwarm): ApiOptions::keepwarm cuts max_tokens, disables retries and marks the telemetry record "keepwarm": true; the transport holds the ping text out of marker placement, so every breakpoint lands on the block the round itself marked and the ping is an uncached tail. A read refreshes the entry’s TTL, so the request after the round reads the prefix the round cached. Nothing is persisted and no history is touched.

Policy (the idle keep-warm’s, session::cache_keepwarm, minus the idle guesswork: the next request is certain unless the turn is cancelled):

• eligible only on the native Anthropic transport and when the round’s request cached something;
• due at anchor + ping_interval(honoured TTL) (4.5 min on 5m, 55 min on 1h) on whichever clock comes first; never once the cache is within LATE_GUARD of expiry (it is presumed gone: the next request rewrites it anyway, a ping would only add a write);
• ski-rental bound: a ping is sent only while ledger + expected ping ≤ rewriting the prefix at the honoured TTL;
• the turn’s max_cost_usd (if any) must admit the ping’s worst case;
• stop on a ping that read nothing, wrote or paid uncached for more than SHARE_LIMIT_PERCENT of the prefix, a 4xx, or exhausted transient retries (BACKOFF).

Stopping. When the tools return, no new ping is scheduled, but a ping already in flight is allowed to finish — for up to FINISH_GRACE — and is booked like any other before the next request goes out. Only a turn cancel, the stream ending, or the grace running out aborts one in flight. Either way the ping is settled before the round’s next request is sent, so its usage never lands after that request’s. (Provider-side order would not matter anyway: an aborted ping and the next request read the same prefix, and a ping writes nothing — its tail is uncached.)

Booking. Every ping that may have been billed is booked into the turn, by the idle keep-warm’s rule (keepwarm::ping_booking): with measured usage the transport records it in the turn’s usage counters (turn budget) and its Usage event is forwarded to the turn’s event channel (session / worker cost); a ping dispatched with no usage seen (aborted in flight, a read timeout) is booked at its worst case — the whole prefix read, PING_INPUT_ESTIMATE uncached, max_tokens out — through both; one refused before the network or rejected with an HTTP error books nothing.

Structs

ToolKeepWarm

Per-turn settings: present only when cache_keepwarm_tools is on.

Constants

FINISH_GRACE

How long the end of a tool round waits for a ping already in flight before aborting it (and booking it at its worst case). Longer than every keep-warm ping in the api-log so far (122 real pings: p50 1.9 s, p95 11 s, max 15.8 s): the wait only happens when a ping straddles the end of the round (a few seconds out of every 270 s interval), and it delays the next request by at most the rest of that ping.

Functions

interval_override

Test seam replacing ping_interval(honoured TTL) for tool-round pings. Separate from the idle keep-warm’s seam so tests of one never fire the other. A constant None without cfg(test) / the testing feature; no binary input (env, config, argv) can reach it.