Skip to main content

Module cache_health

Module cache_health 

Source
Expand description

Session-level prompt-cache health monitoring.

The harness already warns about individual cache-breaking events (reasoning effort changes, idle-gap expiry, planning transitions). This module adds the missing aggregate view: it observes normalized per-turn usage and fires a session-scoped alert when the hit rate stays degraded, mirroring the “monitor cache hit rate like uptime” discipline. Both runloops (the vtcode-core AgentRunner and the binary unified runloop) feed the same monitor through their usage-merge paths so thresholds and wording stay identical.

The monitor consumes the canonical harness Usage (as produced by normalized_turn_usage), where input_tokens is always the total prompt volume and cached_input_tokens/cache_creation_tokens carry the cache signal, so provider reporting differences are already normalized away.

Structs§

PromptCacheHealthMonitor
Rolling prompt-cache health for one session.

Enums§

CacheHealthAlert
Session-scoped prompt-cache health alert.

Constants§

DEGRADED_TURN_HIT_RATE_PERCENT
A turn counts as degraded when its cache hit rate is below this floor. Partial reuse still counts as healthy; only turns that re-pay (nearly) everything move the needle.
HIT_RATE_FLOOR_PERCENT
Hit-rate floor (percent) that fires the low-hit-rate alert.
HIT_RATE_WINDOW_TURNS
Measured turns before the hit-rate alert arms.
MIN_INPUT_TOKENS_FOR_SIGNAL
Minimum input tokens for a turn to count as cache signal.
SUSTAINED_MISS_TURNS
Consecutive degraded turns that fire the sustained-miss alert.