Enum GatingFunction 

pub enum GatingFunction {
    Softmax,
    Sigmoid,
    SqrtSoftplus,
}

Top-k softmax router over per-expert logits, as used by DeepSeek / GLM / Kimi-style MoE layers (a linear gate scores every expert, top-k experts are kept, and their scores are renormalized to sum to one). Which function converts a router’s raw per-expert logits into selection scores, before top-k selection and normalization.

This distinction is not cosmetic: reading ik_llama.cpp’s actual GGUF-loading source (llama-hparams.cpp) directly showed that DeepSeek-2/3-family models (LLM_ARCH_DEEPSEEK2) and newer GLM-MoE-family models (LLM_ARCH_GLM4_MOE) both default to sigmoid gating with post-selection score normalization, not softmax – matching DeepSeek-V3’s own published technical report, which documents computing per-expert affinity via sigmoid and then normalizing the selected experts’ scores to sum to one. Only older DeepSeek-2.0/2.5-era models default to softmax. Using softmax unconditionally, which ferrox did before this was found, would silently produce wrong routing decisions for any model in the DeepSeek-3/GLM4-MoE lineage – which very plausibly includes DeepSeek V4 Pro and GLM-5.2, both presumed continuations of these architecture families (see docs/MODELS.md for the exact confidence level on this).

Variants

Softmax

exp(logit) / sum(exp(selected logits)) – the older convention.

Sigmoid

sigmoid(logit) for scoring and top-k selection, then the selected sigmoid scores are renormalized to sum to one – the DeepSeek-V3 / GLM4-MoE convention.

SqrtSoftplus

sqrt(softplus(logit)), i.e. sqrt(ln(1 + exp(logit))) – DeepSeek V4’s real MoE scoring function. Confirmed two ways from llama.cpp PR #24162 (src/models/deepseek4.cpp): (1) load_arch_hparams hard-throws ("DeepSeek-V4 loader currently expects sqrtsoftplus MoE scoring") unless the GGUF’s expert_gating_func metadata is exactly LLAMA_EXPERT_GATING_FUNC_TYPE_SQRT_SOFTPLUS; (2) llm_graph_context::build_moe_ffn’s real scoring switch computes probs = ggml_sqrt(ctx0, ggml_softplus(ctx0, logits)) for that enum case. This supersedes the earlier sigmoid guess this crate carried (inherited from the DeepSeek-2/3 lineage), which the real loader source shows is wrong for V4 specifically.

Trait Implementations

impl Clone for GatingFunction

fn clone(&self) -> GatingFunction

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for GatingFunction

impl Debug for GatingFunction

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for GatingFunction

impl PartialEq for GatingFunction

fn eq(&self, other: &GatingFunction) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for GatingFunction