Struct MoeWeights 

pub struct MoeWeights {
    pub router: WeightMatrix,
    pub experts: ExpertBacking,
    pub shared_experts: Vec<ExpertWeights>,
    pub shared_expert_gate: Option<Vec<f32>>,
    pub norm_weight: Vec<f32>,
    pub exp_probs_bias: Option<Vec<f32>>,
    pub activation_counts: Vec<AtomicU64>,
}

Fields

router: WeightMatrix

experts: ExpertBacking

shared_experts: Vec<ExpertWeights>

shared_expert_gate: Option<Vec<f32>>

Qwen2-MoE-specific: when present, the shared experts’ combined output is scaled by sigmoid(shared_expert_gate . x) before being added to the routed output, instead of added unconditionally – confirmed against the real transformers source (Qwen2MoeSparseMoeBlock.forward: shared_expert_output = F.sigmoid(self.shared_expert_gate(hidden_states)) * shared_expert_output) and llama.cpp’s real qwen2moe.cpp (ffn_gate_inp_shexp dotted against the hidden state, sigmoid, multiplied into the shared-expert branch before the final add). Real on-disk shape is [hidden_dim] (a Linear(hidden_dim, 1, bias=false)’s weight, flattened – ggml’s real create_tensor call declares it as {n_embd}, not a 2D matrix), so this is a plain owned vector dotted with the normed hidden state directly, not a WeightMatrix. None for every other architecture (DeepSeek-V3’s shared experts, for one real confirmed contrast, add unconditionally with no gate at all).

norm_weight: Vec<f32>

exp_probs_bias: Option<Vec<f32>>

DeepSeek-V3’s aux-loss-free expert-selection bias, on disk as blk.{N}.exp_probs_b.bias (llama.cpp’s LLM_TENSOR_FFN_EXP_PROBS_B – note the on-disk name has no ffn_ prefix, llama-arch.cpp:416). It is added to the selection score only: the top-k is taken over gating(logit) + bias[expert], while each winner’s combine weight comes from the unbiased gating(logit) (build_moe_ffn: “leave probs unbiased as it’s later used to get expert weights”). Biasing the weight too would silently skew every routed contribution away from what the router learned.

None for every checkpoint that does not ship the tensor. When it is present, the GPU MoE fast paths refuse the layer rather than route without it – their kernels have no bias input.

activation_counts: Vec<AtomicU64>

How many times each routed expert (index into experts) has been selected by route_top_k across every forward_token/ forward_batch call so far. Real observed hotness, not a placeholder – feeds placement_plan below, which is what PlacementPlan::from_budget needs to prioritize actually-hot experts for GPU residency instead of guessing by index.

Implementations

impl MoeWeights

pub fn n_experts(&self) -> usize

pub fn expert_bytes(&self, e: usize) -> usize

This routed expert’s weight byte footprint, from resident matrices or the stored layout – identical numbers either way, so residency planning is backing-independent.

pub fn with_expert<R>(&self, e: usize, f: impl FnOnce(&ExpertWeights) -> R) -> R

Runs f against expert e’s weights, materializing them from the store first when this layer is store-backed. The lease (and therefore the cache entry’s pin) lives exactly as long as f’s borrow.

pub fn placement_plan(&self, vram_budget_bytes: u64) -> PlacementPlan

A real VRAM-budget-and-hotness-driven placement plan for this layer’s routed experts, built from each expert’s actual resident byte size (WeightMatrix::resident_bytes() summed across its gate/up/down matrices, so it reflects the real quantization format in use, not an estimate) and the activation counts observed so far. See ferrox_moe::PlacementPlan::from_budget.