Module gdn 

Qwen-style Gated Delta Net (GDN) — linear-attention / SSM recurrent primitive for hybrid arches (qwen35, qwen35moe, qwen3next, …).

Distinct from Kimi KDA (kda.rs): GDN uses a fused QKV projection, a single depthwise ssm_conv1d over the concatenated QKV channels, per-head ssm_alpha / ssm_beta gates, and decay exp(softplus(α + ssm_dt) · ssm_a) (GGUF ssm_a is typically -exp(A_log)). KDA is not a drop-in for this graph.

GQA geometry (the shape rule this module implements)

GDN is GQA-shaped: a real checkpoint has fewer K heads than V heads (num_value_heads % num_key_heads == 0) and the K and V head dims need not be equal (key_head_dim ≠ value_head_dim is legal). Transcribed from FreeToken’s qwen3_5_moe/gdn_reference.py (Qwen3_5GatedDeltaNetReference.forward, no-cache path) and LinearGatedDeltaGroupConfig, where num_key_heads / num_value_heads / key_head_dim / value_head_dim are four independent numbers, not two:

• Split offsets. The fused projection produces conv_dim = 2·key_dim + value_dim channels, with key_dim = num_key_heads · key_head_dim and value_dim = num_value_heads · value_head_dim, and is split as [key_dim, key_dim, value_dim] — not into three equal thirds. Assuming equality computes the K and V offsets wrong, so every head reads a slice straddling the wrong tensor: the layer stays finite and correctly shaped and silently returns garbage rather than failing.
• Replication. Each K head’s q/k pair is replicated num_value_heads / num_key_heads times (repeat_interleave on the head axis) before the recurrence, so V head h reads K head h / rep. Skipping the replication indexes q/k past the end of the Q slice (into K, then into V) instead of reusing the shared head.
• Rectangular state. The recurrent state is [num_value_heads, key_head_dim, value_head_dim], not square, and the q scale is key_head_dim^-0.5 (the key dim — the reference takes dk from key.shape[-1]). Sizing the state from one head dim under-allocates whenever key_head_dim > value_head_dim and mis-strides the read-out in either direction.

With num_key_heads == num_value_heads and key_head_dim == value_head_dim every rule above collapses to the older equal-head path, bit for bit — pinned by equal_head_geometry_stays_bit_identical_to_the_pre_generalization_output.

GGUF tensor name mapping (per layer L)

Role	GGUF name
Fused Q‖K‖V	blk.{L}.attn_qkv.weight
Output / z gate	blk.{L}.attn_gate.weight
Depthwise causal conv	blk.{L}.ssm_conv1d.weight
Decay bias	blk.{L}.ssm_dt.bias (alt: ssm_dt)
Decay scale	blk.{L}.ssm_a
Input gate β	blk.{L}.ssm_beta.weight
Forget raw α	blk.{L}.ssm_alpha.weight
Output RMSNorm	blk.{L}.ssm_norm.weight
Output projection	blk.{L}.ssm_out.weight

Legacy qwen3next may pack β/α into ssm_ba or fuse QKV+z into ssm_in; this module implements the split qwen35 layout only.

GGUF weight load skeleton: crate::hybrid_gguf_loader. Serve still fail-closed — factory HybridEngine::reject.

Structs

GdnConfig

Dims for one Qwen35-style GDN block, with independent K/V head counts and K/V head dims (see the module docs for why all four are separate numbers).

GdnState

Fixed-size recurrent + short-conv state (unlike growing KV).

GdnWeights

Weights matching the qwen35 GGUF layout (see module docs).

Functions

gdn_forward_token

One decode step of the GQA-shaped Gated Delta Net.