Module dequant_cache 

Cache dequantized GGUF weight bytes for static params.

Qwen3.5 decode with --packed was re-dequantizing every K-quant weight on every matmul (hundreds of times per token). Keys are (w_off, len, k, n, scheme, sample) — arena byte offset plus a cheap head/mid/tail sample so distinct tensors that share an offset across graphs do not collide. Do not key on the absolute data pointer: the CPU arena base can move between forwards, which would treat every token as a cache miss and OOM after duplicating f32 weights.

Functions

cache_thrashing

Whether the cache is THRASHING: the f32 working set does not fit the budget, so entries are evicted before they are ever reused and every lookup pays a full re-dequant.

clear_dequant_cache

Drop cached dequantized weights (e.g. between model loads in tests).

gguf_weight_f32

Return dense [k×n] weights (GGUF row-major [n,k] layout) for w_bytes.