Enum GraphFfn 

pub enum GraphFfn<'a> {
    Dense {
        gate: GraphW<'a>,
        up: GraphW<'a>,
        down: GraphW<'a>,
    },
    Moe {
        router: GraphW<'a>,
        shared_gate: GraphW<'a>,
        experts: Vec<(usize, usize, usize)>,
        n_exp: usize,
        top_k: usize,
        inter: usize,
        norm_topk: bool,
        q4tp: bool,
        gu_q2: bool,
        sigmoid: bool,
        bias: Option<&'a [f32]>,
        has_shared: bool,
    },
}

The FFN of one graph layer: a dense SwiGLU trio, or a routed MoE — router + top-k selection + all selected experts run ON DEVICE (the routing decision depends on the resident hidden state, so a CPU round-trip per layer would forfeit the one-submit design).

Variants

Dense

Fields

gate: GraphW<'a>

up: GraphW<'a>

down: GraphW<'a>

Moe

Fields

router: GraphW<'a>

Router logits weight (f32, kind 4) [n_exp, hidden].

shared_gate: GraphW<'a>

Shared-expert sigmoid gate (f32) [1, hidden].

experts: Vec<(usize, usize, usize)>

Per-expert q4_tiled directory indices (gate, up, down); the SHARED expert rides as the LAST entry — the select kernel pins it with the sigmoid weight.

n_exp: usize

Routed experts (shared excluded).

top_k: usize

inter: usize

norm_topk: bool

q4tp: bool

Expert weight layout, uniform across the layer: false = q4_tiled (18 B tiles, inline f16 scale), true = q4tp (16 B nibbles + a per-row ladder plane). The two differ only in where the scale comes from, so they share every kernel but the weight-staging block.

gu_q2: bool

true = the gate/up experts are q2tp (2-bit plane) while down stays q4tp — the mixed profile a 2-bit-class checkpoint converts into. Only meaningful with q4tp: true.

sigmoid: bool

LFM2-MoE / DeepSeek-V3 noaux_tc routing: per-expert sigmoid scores instead of a softmax, and norm_topk renormalises with the 1e-6 floor. The softmax arm is bit-identical to before.

bias: Option<&'a [f32]>

Per-expert SELECTION bias: added to the score for the top-k choice only — the mixing weights stay unbiased (noaux_tc).

has_shared: bool

Whether a shared expert rides as the last experts entry. LFM2-MoE has none; the select kernel then leaves slot top_k unwritten and the expert loop runs top_k slots, not +1.