Struct TransformerConfig 

pub struct TransformerConfig {
    pub num_layers: usize,
    pub hidden_size: usize,
    pub num_attention_heads: usize,
    pub num_kv_heads: usize,
    pub head_dim: usize,
    pub intermediate_size: usize,
    pub vocab_size: usize,
    pub max_seq_len: usize,
    pub rms_norm_eps: f32,
    pub has_qk_norm: bool,
}

Configuration for a standard transformer decoder.

Fields

num_layers: usize

hidden_size: usize

num_attention_heads: usize

num_kv_heads: usize

head_dim: usize

intermediate_size: usize

vocab_size: usize

max_seq_len: usize

rms_norm_eps: f32

has_qk_norm: bool

Whether Q/K heads have per-head normalization (Qwen3 has this, Llama doesn’t).

Trait Implementations

impl Clone for TransformerConfig

fn clone(&self) -> TransformerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TransformerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.