Struct MoeWeights 

pub struct MoeWeights {
    pub router: WeightMatrix,
    pub experts: ExpertBacking,
    pub shared_experts: Vec<ExpertWeights>,
    pub shared_expert_gate: Option<Vec<f32>>,
    pub norm_weight: Vec<f32>,
    pub activation_counts: Vec<AtomicU64>,
}

Fields

router: WeightMatrix

experts: ExpertBacking

shared_experts: Vec<ExpertWeights>

shared_expert_gate: Option<Vec<f32>>

Qwen2-MoE-specific: when present, the shared experts’ combined output is scaled by sigmoid(shared_expert_gate . x) before being added to the routed output, instead of added unconditionally – confirmed against the real transformers source (Qwen2MoeSparseMoeBlock.forward: shared_expert_output = F.sigmoid(self.shared_expert_gate(hidden_states)) * shared_expert_output) and llama.cpp’s real qwen2moe.cpp (ffn_gate_inp_shexp dotted against the hidden state, sigmoid, multiplied into the shared-expert branch before the final add). Real on-disk shape is [hidden_dim] (a Linear(hidden_dim, 1, bias=false)’s weight, flattened – ggml’s real create_tensor call declares it as {n_embd}, not a 2D matrix), so this is a plain owned vector dotted with the normed hidden state directly, not a WeightMatrix. None for every other architecture (DeepSeek-V3’s shared experts, for one real confirmed contrast, add unconditionally with no gate at all).

norm_weight: Vec<f32>

activation_counts: Vec<AtomicU64>

How many times each routed expert (index into experts) has been selected by route_top_k across every forward_token/ forward_batch call so far. Real observed hotness, not a placeholder – feeds placement_plan below, which is what PlacementPlan::from_budget needs to prioritize actually-hot experts for GPU residency instead of guessing by index.

Implementations

impl MoeWeights

pub fn n_experts(&self) -> usize

pub fn expert_bytes(&self, e: usize) -> usize

This routed expert’s weight byte footprint, from resident matrices or the stored layout – identical numbers either way, so residency planning is backing-independent.

pub fn with_expert<R>(&self, e: usize, f: impl FnOnce(&ExpertWeights) -> R) -> R

Runs f against expert e’s weights, materializing them from the store first when this layer is store-backed. The lease (and therefore the cache entry’s pin) lives exactly as long as f’s borrow.

pub fn placement_plan(&self, vram_budget_bytes: u64) -> PlacementPlan

A real VRAM-budget-and-hotness-driven placement plan for this layer’s routed experts, built from each expert’s actual resident byte size (WeightMatrix::resident_bytes() summed across its gate/up/down matrices, so it reflects the real quantization format in use, not an estimate) and the activation counts observed so far. See ferrox_moe::PlacementPlan::from_budget.