Module context_cache 

Client-side prefix cache hit estimation.

The provider reports cache_read_tokens for real prompt-cache reads, but there is no provider-agnostic signal for how much of the final context prefix actually overlapped with the previous request. This module implements a lightweight, tokenizer-free estimate:

1. Serialize the final provider Context into a canonical byte sequence.
2. Split the bytes into fixed-size chunks and hash each chunk.
3. Compare the current chunk list with the previous request’s list from index 0 (longest common prefix).
4. Convert overlapping bytes to tokens using the provider-reported total input token count as the byte-to-token calibration.

The estimate is intentionally approximate and only intended to explain cache trends, not to replace provider-reported cache accounting.

Structs

ContextCacheTracker

Per-session, per-model prefix overlap tracker.

PrefixHitEstimate

Result of comparing the current context against the previous baseline.

PrefixHitResult

Final prefix-hit metrics after provider usage is available.

Constants

CONTEXT_CHUNK_SIZE

Chunk size for the prefix-overlap comparison.

Functions

canonical_context_bytes

Canonical, deterministic byte representation of the final provider context.