Struct GdnConfig 

pub struct GdnConfig {
    pub hidden_dim: usize,
    pub num_key_heads: usize,
    pub num_value_heads: usize,
    pub key_head_dim: usize,
    pub value_head_dim: usize,
    pub conv_kernel_size: usize,
    pub rms_norm_eps: f32,
}

Dims for one Qwen35-style GDN block, with independent K/V head counts and K/V head dims (see the module docs for why all four are separate numbers).

Invariant: num_key_heads > 0 and num_value_heads % num_key_heads == 0. The quotient is the q/k replication factor; a config that violated it would leave V heads with no K head to read from, so gdn_forward_token asserts it instead of flooring the division and silently pairing a V head with a K head that never fed it.

Fields

hidden_dim: usize

num_key_heads: usize

Number of Q/K heads in the checkpoint (≤ num_value_heads).

num_value_heads: usize

Number of V heads — also the length of ssm_dt / ssm_a and the row count of ssm_beta / ssm_alpha.

key_head_dim: usize

Width of one Q/K head (dk; sets the dk^-0.5 q scale).

value_head_dim: usize

Width of one V head (dv; also the width of ssm_norm).

conv_kernel_size: usize

rms_norm_eps: f32

Implementations

impl GdnConfig

pub fn key_dim(&self) -> usize

Channels occupied by Q (and, separately, by K) in the fused projection: num_key_heads · key_head_dim.

pub fn value_dim(&self) -> usize

Channels occupied by V in the fused projection — also the width of the attn_gate (z) projection and of ssm_out’s input.

pub fn qkv_dim(&self) -> usize

Fused Q‖K‖V width: key_dim + key_dim + value_dim.

This is the number the equal-head formula got wrong: 3 · num_value_heads · head_dim only coincides with the real width when both head counts and both head dims match. It is the same arithmetic that yields the split offsets, so a wrong total means wrong Q/K/V slices — not a length mismatch anyone would notice.

pub fn heads_per_key_group(&self) -> usize

q/k replication factor: how many V heads share one K head (the repeat_interleave count in the reference). 1 for the equal-head geometry.

Trait Implementations

impl Clone for GdnConfig

fn clone(&self) -> GdnConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for GdnConfig

impl Debug for GdnConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.