Structs§
- Compressed
Attention Hit - One selected compressed-attention hit from the shared cold pool.
- Compressed
Attention Selection - Output of compressed-domain top-k attention selection over the cold pool.
- Decompressed
Layer - Reconstructed K/V tensors for one layer of the shared pool.
- Fully
Prepared Compressed Index - Fully prepared compressed index: pre-unpacks all indices and norms so the scoring loop is just Gram table lookups with no per-call unpacking.
- Pool
Layer - One layer’s worth of compressed KV blocks in the shared pool.
- Prefetched
Gram Rows - Pre-fetched Gram rows for a specific query, enabling cache-friendly scoring.
- Prepared
Compressed Index - Pre-built compressed index for one layer/head, avoiding per-call reconstruction.
- SharedKV
Pool - A shared, compressed KV cache pool.
Traits§
- Cache
Target - Trait for KV cache targets that can receive injected blocks.