Expand description
Device-side homes for the expert slot pool
crate::expert_slots::ExpertSlots governs.
crate::expert_cache decides which experts are resident and validates the
plans that make them so, but holds no device memory by design. This
is the other side of that line: the allocations, and the
[SlotDevice] implementations that write into them.
§What is verified, and what is not
The policy is verified: every rule about plans, occupancy,
attribution of failures and the zero-copy warm step is tested in
crate::expert_slots on any host.
[CudaExpertPool] is compile-verified only. ferrox holds CUDA
to a must-compile bar and its hardware tests stay #[ignore]d, and
no benchmark host has run this. It is written out rather than
stubbed because its correctness is mostly the type system’s to
check – an allocation per slot, a bounds-checked index, a driver
copy – unlike a timing loop, where writing one without a machine to
run it on would put a number nobody measured into a profile. Where
this file makes a performance claim it says so is unmeasured.
§One allocation per slot, not one per bank
The obvious layout is one contiguous slots * row_bytes buffer per
bank. This allocates each slot separately instead, for a reason
that is not aesthetic: a slot-to-slot copy needs a shared borrow of
the source and a mutable borrow of the destination at the same
time, and two sub-views of one CudaSlice cannot provide that.
Separate allocations can, via split_at_mut, so the gather path is
ordinary safe code rather than raw driver pointer arithmetic
written against hardware nobody here can run.
The cost is slots * banks allocations at startup instead of
banks. It is paid once, and it buys back the thing each slot is
for: one expert row, one device pointer, which is exactly what a
matvec launch wants.
Functions§
- split_
pair - Borrows two distinct slots of one bank at once: the source shared, the destination mutable.