Expand description
Cache-aligner volatile-field detection (#940, Headroom “cache aligner” stage
- — telemetry-first.
A stable system prompt is the largest prefix a provider can cache, but a
single turn-to-turn-varying token inside it (today’s date, a fresh UUID, a
git SHA) shifts the bytes and busts the cache on every request. Headroom’s
cache aligner relocates those volatile fields to the tail so the prefix
stays byte-stable. Relocating provider-visible system content is risky, so
this phase ships only the measurement half: a deterministic detector that
counts the volatile fields in an unanchored system prompt, surfaced on
/status so a user can see how much cache their prompt is leaking before any
opt-in relocate is enabled.
§Why measure first
The honest, low-risk order is: detect → quantify (telemetry) → only then offer
an opt-in tail-relocate behind its own flag, once the data shows it pays. The
relocate, when added, will reuse the stable-first ordering of
crate::core::neural::cache_alignment::CacheAlignedOutput (today only
exercised by the doctor self-test) as its building block.
§Determinism (#498)
The scan is a pure function of the text: every pattern’s matches are collected, sorted, and overlapping spans merged, so the field count and covered-byte total are stable across runs and never depend on hash-map order. It mutates nothing — the request body is byte-identical whether the scan runs or not.