Module kv_block 

Content-addressed identity for KV-cache blocks: what makes two stored prefixes the same prefix, across processes and across restarts.

A block is a fixed-size run of token positions in one sequence. Its identity is a parent-chained SHA-256:

root       = H(domain, "root",  model, extra_keys)
block[i]   = H(domain, "block", model, extra_keys, parent, token_ids)
parent     = root for block[0], block[i-1] otherwise

Chaining is what makes a hash mean this token run, at this offset, after exactly this history rather than merely “these tokens”. Without it, two different prompts that happen to share an interior token run would collide on a block whose KV state depends on everything before it, and the cache would hand back state computed under a different history – silent wrong answers, not a miss.

extra_keys is the salt slot for anything that changes what the KV state means without changing the token ids: a LoRA adapter’s identity, an image/audio embedding’s identity for a multimodal prompt. Sampling parameters are deliberately not part of the key: KV state is sampling-independent, and folding temperature or a seed into the key would only shatter the cache.

Every field is length-prefixed before hashing, so no two different (model, extra_keys, parent, tokens) tuples can serialize to the same byte string. Concatenating raw fields would let extra_keys = ["ab", "c"] and ["a", "bc"] hash identically.

This module is identity only – no storage, no eviction, no I/O. The disk tier that will consume it is kv-ssd-tier in docs/plans/serving-and-tiered-kv.md.

Structs

BlockHash

A block’s content address: the 32-byte SHA-256 of its chained identity.

BlockHasher

Hashes blocks for one (model, extra_keys) identity. Cheap to build; the root is computed once and reused for every chain.

Functions

full_blocks

How many whole blocks token_count tokens make at block_size.