Module expert_slots 

The executor for crate::expert_cache’s plans: a bounded slot pool, the copies a plan asks for, and the counter that says a warm step moved nothing.

Why this module exists

ExpertCache decides which experts are resident and returns a CopyPlan – and that is where it stops, deliberately: “nothing here moves bytes; the plan is the whole output”. Until something executes those plans the whole q* split is inert, because a policy that decides how much to fetch decides nothing while no fetch happens. This is the other half: it takes a plan, validates it against a pool it owns the geometry of, performs exactly the copies the plan names, and records what crossed the link.

Still no device memory here

The bytes live behind SlotDevice, so this crate keeps holding no tensors and no device allocations, and every rule below stays testable on a host with no GPU at all. HostSlotMemory is a real implementation of that trait rather than a test mock – it is the pool a CPU-only build uses, where the “link” is a memcpy.

A plan is applied whole or not at all

Every check happens in a pre-pass, before a single byte is written. A plan is a unit: the cache has already recorded the residency the plan describes by the time the caller gets it, so a half-applied plan leaves the cache claiming an expert lives in a slot that holds someone else’s bytes. That does not fail – it multiplies, and returns a confident wrong answer. Refusing up front keeps the device exactly as it was, which is a state the cache can be told to resync to.

A device fault is the one thing that can still land mid-plan, since only the device knows it failed. Whatever it cost is marked unknown here and named in the error, so the caller can hand the same slots to ExpertCache::forget_slot and have the next step re-fetch them instead of reading them. How much is suspect depends on when it failed: a refused copy costs its own slot (SlotFault::Device), while a failed flush costs every slot the plan wrote (SlotFault::DeviceFlush) – a backend that batches its transfers cannot say which of them landed.

Row size is checked exactly, not as a minimum

A host row shorter than the slot would leave the slot’s tail holding the previous occupant’s bytes: a coherent-looking expert spliced from two, which produces plausible tokens rather than an error. So a length mismatch in either direction is refused.

Structs

Applied

What one applied plan moved.

ExpertSlots

A bounded pool of expert slots, and the executor for the plans that fill it.

HostSlotMemory

A pool whose “device” is host memory.

SlotGeometry

The shape of one pool: how many slots, how many banks, and how wide a row is in each bank.

SlotStats

Running totals for /metrics and for the A/B that says whether residency is paying for itself.

Enums

SlotFault

Why a plan was refused, or how applying it failed.

Traits

ExpertRows

The host-side expert rows a CopyPlan reads from.

SlotDevice

Where a slot’s bytes actually live.