Module keepwarm 

Prompt-cache keep-warm ping (task #444).

Anthropic refreshes a cache entry’s TTL every time it is read (“hits and refreshes” are billed at the cache-read rate), and the lifetime counts from the START of the request that read or wrote it. Re-sending the session’s exact prefix shortly before the TTL lapses keeps a large prefix warm at the read price instead of paying a full rewrite when the user comes back. The policy (when to ping, when to stop, what it costs) lives in the session actor (session::cache_keepwarm); this module builds, fingerprints and sends one ping.

The ping is the request the NEXT real turn would send, minus the parts that are per-turn by construction, and goes through the SAME request builder (ApiMethods::call_api_stream_inner → build_anthropic_request):

• tools, system prompt (orchestration doctrine, frozen tool catalog, context guidance, session-scoped extension injection), thinking config, effort, model, TTL markers and the durable history are assembled exactly as run_stream_tracked + run_stream_internal assemble them;
• the trailing message is a fixed user text block, appended request-locally AFTER the last cache marker (ApiOptions::keepwarm moves the conversational marker onto the last history message), so it never enters the cached prefix and is never persisted;
• no memory recall, no before_message / Inject hooks, no steering, no context-pressure message: those only ever touch the request tail;
• no MessageHistory, no turn, no transcript: events go to a private channel and only the Usage numbers come back;
• max_tokens is cut to the minimum the thinking mode allows (RequestBody::apply_keepwarm_max_tokens; not part of the cache key), retries are zero, and the telemetry record says "keepwarm": true.

Prefix fingerprint. A ping is only correct while the next real turn’s prefix is byte-identical to what is cached; on a changed prefix it would WRITE a new entry instead of reading the old one. The builder hashes the serialized prefix (RequestBody::prefix_fingerprint) of every keep-warm request. Runtime::cache_keepwarm_probe returns that hash without sending anything; Runtime::cache_keepwarm_ping takes the expected hash and refuses to send — inside the builder, after assembly, right before the network — when the freshly built prefix differs.

Structs

CacheAnchor

When this session’s runtime last had an Anthropic request accepted (HTTP 2xx): the moment its cache entries were last written or refreshed. Taken immediately BEFORE the bytes are handed to the HTTP client, so it is never later than the provider’s own request start (the TTL origin).

CacheAnchorCell

Shared “last accepted Anthropic request” slot (see CacheAnchor). Runtime clones get a FRESH cell (a clone is a different request chain, like the TTL latches); share_ttl_latches shares it.

KeepWarmProbe

What a probe learned about the next ping, without sending anything.

KeepWarmUsage

Token usage of one ping, as reported by the provider.

PingBill

A ping’s booking (see PingTap::bill).

PingReport

One ping’s result AND whatever the provider billed for it. Usage is returned on every path — success, transient or fatal error, in-stream error under a 200 — so the caller books it all.

PingTap

Live view of an in-flight ping, shared with the actor so an ABORTED ping (a turn started, the session ended) can still be booked: sent flips once the request is dispatched (the HTTP client pulls the body over an established connection, or a response arrives — never for a refused connection), connect_failed when send() could not connect, status holds the last HTTP status (0 = none), and usage accumulates Usage events as they stream in.

Enums

PingBooking

How a finished or aborted ping is booked (see ping_booking).

PingFailure

Why a ping failed.

PingOutcome

What a ping did.

Constants

KEEPWARM_OUTPUT_TOKENS

Output tokens a ping may produce on top of a thinking budget (or in total with thinking off / adaptive). One token: the answer is discarded.

KEEPWARM_PING_TEXT

The trailing user message. Sits after the last cache marker, is never persisted, and asks for the shortest possible answer with no reasoning so budgeted thinking spends as little of its budget as the model allows.

Functions

history_ready

Whether history can be kept warm: it must end with a finished assistant reply (a turn that errored or was interrupted leaves nothing worth pinging).

ping_booking

The one booking rule for a ping, on every path: usage arrived → its real cost; otherwise rejected with an HTTP error status (≥ 400) → $0; otherwise dispatched → the worst case, unless the error was a connect failure with no HTTP status; otherwise $0.

reply_prefix_tokens

Conservative token estimate of the part of the history’s last message (the turn’s final reply) that the cached prefix will hold: its content minus thinking blocks, which the API strips when the next request re-sends the reply (they were billed as output, never cached). Ping 1 may write at most this (plus the last request’s uncached input) without counting as a partial miss.