Module glm52_decoder 

A dedicated decoder for GLM-5.2’s real architecture, separate from ferrox-models::decoder::Decoder (the generic GQA path every other preset uses) – analogous to kimi_decoder.rs’s role for Kimi K3: composes the already-independently-tested glm_dsa attention module with a standard SwiGLU dense/MoE FFN into a real forward pass, without touching the existing GQA decoder.

Real per-layer flow, transcribed from llama_model_glm_dsa::graph’s real per-layer loop in src/models/glm-dsa.cpp (confirmed against both PR #23346’s DeepSeek-V3.2 fork point and PR #25407’s GLM-5.2 diff on top) – notably simpler than Kimi K3’s real per-layer flow (kimi_decoder.rs’s module doc comment): no block-residual scaffolding at all, an ordinary pre-norm transformer block:

attn_in = rms_norm(hidden, attn_norm)
attn_out = glm52_attn_forward_token(attn_in)   // glm_dsa module
ffn_in = hidden + attn_out
ffn_out = rms_norm(ffn_in, ffn_norm) |> dense_or_moe_ffn
hidden = ffn_in + ffn_out

FFN: dense leading layers use ordinary SiLU-gated SwiGLU (ffn_gate/ffn_up/ffn_down, ggml’s LLM_FFN_SILU/ LLM_FFN_PAR – the same convention every other architecture’s dense FFN uses in this codebase, ferrox_core::matmul::swiglu). MoE layers use real sigmoid gating with an aux-loss-free per-expert bias (noaux_tc, confirmed against the real config.json: "scoring_func": "sigmoid", "topk_method": "noaux_tc" – see docs/MODELS.md) plus a shared expert, reusing ferrox_moe::route_top_k_sigmoid_with_bias/run_expert/ combine_expert_outputs directly rather than re-deriving that math here (it’s already independently tested there, and it’s exactly the same real convention DeepSeek-V3/Kimi K3 use for their own noaux_tc routing).

Not yet run against a real GLM-5.2 checkpoint (~744B params, no feasible download in this environment) – tested here against synthetic weights only, cross-validated for the attention math via glm_dsa’s own independent Python cross-check; this module’s own test additionally confirms the full decoder (attention + both dense and MoE FFN branches, across a full/shared indexer-layer pair) composes into a finite, real forward pass end to end, the same rigor kimi_decoder.rs’s own test applies for Kimi K3.

Structs

Glm52DecodeState

Glm52DecoderConfig

Glm52DecoderLayerWeights

Glm52DecoderWeights

Glm52DenseFfnWeights

The dense leading layer’s feed-forward block (real tensor names blk.{bid}.ffn_{gate,down,up}).

Glm52MoeFfnWeights

One MoE layer’s real weights: routed experts (sigmoid gating + aux-loss-free bias) plus a shared expert, always active.

Enums

Glm52LayerFfn

Functions

glm52_forward_token

One decode step across every layer. prev_top_k is reset to None at the start of this function (per-token-forward-pass scope, see glm_dsa::glm52_attn_forward_token’s doc comment) – it must not be threaded in from a previous token.