Module moe_cache 

EDGE-1 §B: SLRU GPU expert-residency cache (MOE-SLRU-PLAN §B).

Stage-1 moe_ffn re-stages EVERY routed expert EVERY token over PCIe into one scratch slot. The same ~15-20% of experts recur (the “hot expert” mass), so an SLRU residency cache makes the steady-state re-stage count -> ~0. The cache holds N fixed-address GPU slots (never re-allocated, never fragmented), a BlockId -> slot residency table, an SLRU eviction policy (probation + protected segments; the second-miss “ghost” admission filter was measured a net loss in both regimes and removed 2026-07-08 — first-miss admit is the policy) so a one-off cold expert can never evict a genuinely hot one.

THE bit-identity property (MOE-SLRU-PLAN §B.3): a cache HIT and a MISS feed qmatvec_view the same GGUF block bytes — the only difference is whether the memcpy_htod ran. So the cache-hit weight path is byte-for-byte identical to stage-every-token; the §D.2 gate pins this.

Gated behind MEMRA_MOE_CACHE (default off => current stage-every-token behavior).

Structs

BlockId

Residency key: expert ex of layer layer projection proj is a distinct block.

MoeSlotCache

SLRU GPU expert-residency cache. Slots remain fixed-address for the cache lifetime. Uniform models use one class; mixed-layout models may preallocate several exact-capacity classes.

Enums

DispatchSlot

Where a dispatched block landed (always a retained resident slot since the first-miss-admit policy, 2026-07-08 — the transient staging tier went with the ghost filter).

Constants

PROJ_DOWN

PROJ_GATE

Which projection of an expert (gate/up/down are three distinct GGUF blocks per expert).

PROJ_UP