Struct TransformerConfig 

pub struct TransformerConfig {

    pub d_model: usize,
    pub num_heads: usize,
    pub num_kv_heads: usize,
    pub num_layers: usize,
    pub num_dec_layers: usize,
    pub vocab_size: usize,
    pub max_enc_len: usize,
    pub max_dec_len: usize,
    pub ffn_dim: usize,
    pub no_feedforward: bool,
    pub activation: FfnActivation,
    pub rope_theta: f32,
    pub pad_id: u32,
    pub eos_id: u32,
    pub bos_id: u32,
    pub unk_id: u32,
    pub tool_call_id: u32,
    pub tools_id: u32,
}

Mirror of Python TransformerConfig — all hyperparameters for Needle SAN.

Fields

d_model: usize

num_heads: usize

num_kv_heads: usize

num_layers: usize

num_dec_layers: usize

vocab_size: usize

max_enc_len: usize

max_dec_len: usize

ffn_dim: usize

no_feedforward: bool

activation: FfnActivation

FFN activation function (drelu/swiglu/geglu).

rope_theta: f32

RoPE base frequency (Python: rope_theta, default 10000.0).

pad_id: u32

Special token IDs

eos_id: u32

bos_id: u32

unk_id: u32

tool_call_id: u32

tools_id: u32

Implementations

impl TransformerConfig

pub fn head_dim(&self) -> usize

pub fn kv_repeat(&self) -> usize

Number of times each KV head is repeated to cover all Q heads.

pub fn validate(&self) -> Result<(), &'static str>

Validate structural invariants. Returns an error string if violated. Called by the inference engine before constructing a model.

Trait Implementations

impl Clone for TransformerConfig

fn clone(&self) -> TransformerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TransformerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TransformerConfig

fn default() -> Self

Returns the “default value” for a type.