Struct HostExps 

pub struct HostExps {
    pub bytes: HostBuf,
    pub tiers: Option<Vec<HostBuf>>,
    pub qtype: i32,
    pub in_f: usize,
    pub out_f: usize,
    pub n_expert: usize,
    pub row_bytes: usize,
    pub expert_stride: usize,
    pub layouts: Option<Vec<ExpertLayout>>,
    pub macros: Option<Vec<f32>>,
}

Fields

bytes: HostBuf

tiers: Option<Vec<HostBuf>>

SPILLING-PLAN §1.1: per-expert backing tier. None => the layer fits in one bytes store and every expert slices it (the unchanged in-RAM path). Some => per-expert split: the hottest experts are Pinned (Tier 1, fast async DMA), the rest Mmap into the GGUF (Tier 2, disk demand-fault). expert_bytes(e) resolves tiers[e] if present, else slices bytes.

qtype: i32

in_f: usize

out_f: usize

n_expert: usize

row_bytes: usize

expert_stride: usize

layouts: Option<Vec<ExpertLayout>>

Per-expert encoding metadata when experts in this projection do not share one dtype/layout. None preserves the existing uniform slab contract and every resident/fused fast path. Some routes through the per-expert staged/cache path, using each entry’s qtype/row size.

macros: Option<Vec<f32>>

Per-expert post-matmul macro-scale (ModelOpt NVFP4 weight_scale_2, one scalar per expert tensor). None => all 1.0 (GGUF experts; block scales carry everything). The MoE forward folds gate/up macros into the activation epilogue (gs/us) and the down macro into the per-expert accumulate weight.

Implementations

impl HostExps

pub fn load( e: &Engine, g: &GgufFile, name: &str, ) -> Result<Self, Box<dyn Error>>

Load a stacked 3D expert tensor, keeping its quant bytes on the HOST. e supplies the CUDA context for the optional pinned allocation (§C.1). Default storage is pageable Vec<u8> (identical to the prior behavior); pinned is chosen when MEMRA_MOE_PINNED or MEMRA_MOE_CACHE is set.

pub fn load_stacked_split_from_source( e: &Engine, src: &dyn TensorSource, name: &str, row0: usize, row1: usize, ) -> Result<Self, Box<dyn Error>>

Load a STACKED 3D expert tensor (ne=[in_f,out_f,n_expert]) from any source. GGUF stores the experts this way; the source returns the same mmap bytes (GgufSource::find == tensor_data), so the GGUF path is byte-identical to the prior direct-GgufFile loader. (Safetensors stores N 2D tensors instead — those go through load_from_source, which gathers them.) Row-range variant for FUSED stacked tensors (gemma4 ffn_gate_up_exps: gate = rows [0,ff), up = [ff,2ff) per expert — llama-graph view convention). Copies only the range.

pub fn load_stacked_from_source( e: &Engine, src: &dyn TensorSource, name: &str, ) -> Result<Self, Box<dyn Error>>

pub fn load_tiered( e: &Engine, g: &GgufFile, name: &str, ctx: &mut SpillCtx, ) -> Result<Self, Box<dyn Error>>

SPILLING-PLAN §1.1, §2 step 4: load a stacked 3D expert tensor with a PER-EXPERT tier split. Under MEMRA_SPILL_DISK, the hottest experts (greedy in expert order, until the shared pinned budget in ctx is exhausted) get HostBuf::Pinned (Tier 1, fast async DMA); every remaining expert is HostBuf::Mmap into the GGUF (Tier 2, demand-faulted from disk on first H2D). The resulting bytes are bit-identical to the in-RAM path either way — qmatvec_view is untouched.

ctx.file_map is ONE shared MAP_SHARED mmap of the whole GGUF (Arc-cloned per spilled expert), so the 120 expert tensors of a 40-layer MoE never open the file more than once.

pub fn load_from_source( e: &Engine, src: &dyn TensorSource, ggml_exps_name: &str, n_expert: usize, ) -> Result<Self, Box<dyn Error>>

MoE expert GATHER from a TensorSource (the safetensors path; ST-MOE-PLAN §1.3). GGUF stacks all experts into ONE 3D tensor; HF stores them as N separate 2D tensors model.layers.{il}.mlp.experts.{e}.{gate,up,down}_proj.weight. find returns None for the ggml *_exps name on purpose, so the experts are gathered out-of-band here.

PATH A (load-time only, no quantize): each HF 2D expert tensor is dequantized to f32 and the per-expert blocks are concatenated expert-axis-slowest into ONE contiguous buffer — exactly the layout expert_bytes(e) slices and the staged qmatvec_view (qtype=QT_F32) reads. The same expert_stride == out_f*row_bytes invariant as the GGUF path is asserted at the end.

ggml_exps_name is blk.{il}.ffn_{gate,up,down}_exps.weight; it is split to recover il and the proj. n_expert comes from cfg.moe. The HF per-expert literal mlp.experts.{e}.{p}_proj is the qwen3moe / olmoe layout (a future arch with block_sparse_moe.experts.* would need a branch in hf_expert_name).

pub fn macro_scale(&self, e: usize) -> f32

Host byte slice for expert e (the H2D DMA source). Contiguous block, offset honored. Resolves the per-expert tier when spilling is active (tiers Some), else slices the single Per-expert post-matmul macro-scale (1.0 when absent).

pub fn is_uniform_layout(&self) -> bool

pub fn expert_layout(&self, e: usize) -> ExpertLayout

pub fn max_expert_bytes(&self) -> usize

pub fn expert_bytes(&self, e: usize) -> &[u8]

backing store (unchanged in-RAM path). Each tiers[e] is exactly one expert’s stride.

pub fn prefetch_expert_pages(&self, e: usize) -> bool

Hint that expert e will be staged soon. Uniform slabs advise only this expert’s window; mixed/pruned layouts advise the selected per-expert mmap. Returns false for resident or empty buffers and on unsupported kernels; callers always retain the demand-fault fallback.