Enum GraphFfn 

pub enum GraphFfn<'a> {
    AttentionOnly,
    Dense {
        gate: GraphW<'a>,
        up: GraphW<'a>,
        down: GraphW<'a>,
        act: GraphAct,
    },
    Moe {

        router: GraphW<'a>,
        shared_gate: GraphW<'a>,
        experts: Vec<(usize, usize, usize)>,
        n_exp: usize,
        top_k: usize,
        inter: usize,
        act: GraphAct,
        norm_topk: bool,
        q4tp: bool,
        gu_q2: bool,
        q82: bool,
        sigmoid: bool,
        sigmoid_denom_eps: f32,
        bias: Option<&'a [f32]>,
        has_shared: bool,
        detached_shared: Option<GraphMoeDetachedShared<'a>>,
        shared_gated: bool,
        route_scale: f32,
    
},
}

The FFN of one graph layer: a dense SwiGLU trio, or a routed MoE — router + top-k selection + all selected experts run ON DEVICE (the routing decision depends on the resident hidden state, so a CPU round-trip per layer would forfeit the one-submit design).

Variants

AttentionOnly

A singleton attention-only batch graph. Returns the post-attention residual, allowing a dynamic expert bank to own the FFN separately.

Dense

Fields

gate: GraphW<'a>

up: GraphW<'a>

down: GraphW<'a>

act: GraphAct

act(gate)·up. The builders refuse an activation with no arm here (before this field the graph computed SiLU for any dense FFN).

Moe

Fields

router: GraphW<'a>

Router logits weight (f32, kind 4) [n_exp, hidden].

shared_gate: GraphW<'a>

Shared-expert sigmoid gate (f32) [1, hidden].

experts: Vec<(usize, usize, usize)>

Per-expert q4_tiled directory indices (gate, up, down); the SHARED expert rides as the LAST entry — the select kernel pins it with the sigmoid weight.

n_exp: usize

Routed experts (shared excluded).

top_k: usize

inter: usize

act: GraphAct

Exact gate/up activation used by every uniformly packed routed expert (and its in-bank shared expert, if any). The graph builder verifies uniformity before encoding one kernel contract for the layer.

norm_topk: bool

q4tp: bool

Expert weight layout, uniform across the layer: false = q4_tiled (18 B tiles, inline f16 scale), true = q4tp (16 B nibbles + a per-row ladder plane). The two differ only in where the scale comes from, so they share every kernel but the weight-staging block.

gu_q2: bool

true = the gate/up experts are q2tp (2-bit plane) while down stays q4tp — the mixed profile a 2-bit-class checkpoint converts into. Only meaningful with q4tp: true.

q82: bool

true = every expert’s gate, up AND down is q8_2f (int8 body, f16 row scales, f16 column field; Mellum2.1’s quality file). q4tp and gu_q2 are false then. Its own kernels: each expert’s input is multiplied by THAT expert’s column field.

sigmoid: bool

LFM2-MoE / DeepSeek-V3 noaux_tc routing: per-expert sigmoid scores instead of a softmax. sigmoid_denom_eps carries the source architecture’s selected-score renormalization floor.

sigmoid_denom_eps: f32

bias: Option<&'a [f32]>

Per-expert SELECTION bias: added to the score for the top-k choice only — the mixing weights stay unbiased (noaux_tc).

has_shared: bool

Whether a shared expert rides as the last experts entry. LFM2-MoE has none; the select kernel then leaves slot top_k unwritten and the expert loop runs top_k slots, not +1.

detached_shared: Option<GraphMoeDetachedShared<'a>>

A separately evaluated, ungated shared SwiGLU expert. It is deliberately outside experts: those tensors cannot share the routed bank’s stride when their intermediate width differs.

shared_gated: bool

The shared expert carries a sigmoid gate (Qwen2/3-MoE). false with has_shared: the shared expert enters with weight 1 (DeepSeek-V3 / HunYuan hy_v3) and shared_gate is a stand-in the select kernels ignore.

route_scale: f32

Multiplier on the routed mixing weights after the optional renormalization (routed_scaling_factor); 1.0 = none.