Enum RouterInput 

pub enum RouterInput {
    NormedFfnInput,
    RawLayerInput,
    NormedLayerInput,
}

The operand of the MoE router’s matmul.

Variants

NormedFfnInput

gate_inp · ffn_norm(ffn_inp) – the normed FFN input, the same vector the experts read. build_moe_ffn’s own computation, and every generic-path graph but one.

RawLayerInput

gate_inp · inpL – the residual stream as it enters the layer, unnormed, before attention (smallthinker.cpp:111). The experts still read the normed FFN input.

NormedLayerInput

gate_inp · ffn_norm_exps(inpSA) – the residual stream as it enters the layer, normed by blk.N.ffn_norm_exps (REQUIRED, arctic.cpp:45,136-139). The routed experts read the SAME vector; the layer’s dense FFN reads ffn_norm(ffn_inp).

Implementations

impl RouterInput

pub fn experts_read_router_operand(self) -> bool

Whether the routed experts read the router’s operand rather than the normed FFN input: true for Self::NormedLayerInput alone. RawLayerInput’s experts read ffn_norm(ffn_inp) (smallthinker.cpp:151), as the default’s do.

pub fn needs_exps_norm(self) -> bool

Whether the layer carries a ffn_norm_exps weight for the operand.

Trait Implementations

impl Clone for RouterInput

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for RouterInput

impl Debug for RouterInput

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RouterInput

fn default() -> Self

Returns the “default value” for a type.

impl Eq for RouterInput

impl PartialEq for RouterInput

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for RouterInput