pub struct DevExps {
pub gate: CudaSlice<u8>,
pub up: CudaSlice<u8>,
pub down: CudaSlice<u8>,
pub ptr_row: CudaSlice<u64>,
pub dev: usize,
pub gu_il: bool,
}Expand description
Device-resident expert slabs for one layer (gate/up/down) + the prebuilt [3, n_expert] pointer row the _dev kernels consume.
Fields§
§gate: CudaSlice<u8>§up: CudaSlice<u8>§down: CudaSlice<u8>§ptr_row: CudaSlice<u64>[3*n_expert] u64 device row: gate ptrs, up ptrs, down ptrs (proj-major like layer_dev_row).
dev: usizeThe CUDA device ordinal these slabs live on (the OWNING stage’s device under the PP
sharded loader — cx-503b sizes and layer_engine places per device). Consumers that
dispatch from a DIFFERENT device must NOT dereference the slabs: an m=1 qmatvec over
peer-read expert bytes is the measured 34-150x slow class (research/pp-prefill-20260807
anatomy), strictly worse than SLRU staging. The sequential arm’s slab-locality gate
(lane/pp-leverb) keys on this field; the per-stage prime walker makes every layer’s
slab local by construction.
gu_il: boolWALL-GAP ARC (MEMRA_MOE_GU_IL=1): gate/up rows INTERLEAVED in one slab — row o of gate at base + o*(rb_g+rb_u), up at +rb_g. Consumers on the dev path must use (rb_g+rb_u) as the row stride for BOTH projections (see MoeWeights::dev_rb_gu). One contiguous 1760B stream per (expert,row) instead of two scattered 880B streams — the measured 56%-of-wall fix candidate. Kernels unchanged (stride is already a parameter everywhere).