Type Alias LlamaConfig 

pub type LlamaConfig = Lfm2Config;

Alias kept for older code that used the Llama-specific config name.

Aliased Type

pub struct LlamaConfig {

    pub hidden_size: usize,
    pub intermediate_size: usize,
    pub num_attention_heads: usize,
    pub num_hidden_layers: usize,
    pub vocab_size: usize,
    pub norm_eps: f32,
    pub block_dim: Option<usize>,
    pub block_ff_dim: Option<usize>,
    pub layer_types: Vec<String>,
    pub num_key_value_heads: Option<usize>,
    pub rope_theta: Option<f64>,
    pub quantization: Option<QuantizationConfig>,
    pub bos_token_id: Option<u32>,
    pub eos_token_id: Option<u32>,
    pub attention_dropout: Option<f32>,
    pub model_type: Option<String>,
    pub tie_word_embeddings: Option<bool>,
    pub head_dim: Option<usize>,
}

Fields

hidden_size: usize

Width of the residual stream.

intermediate_size: usize

Width of the feed-forward (MLP) hidden layer.

num_attention_heads: usize

Number of attention heads.

num_hidden_layers: usize

Number of transformer layers.

vocab_size: usize

Number of tokens in the vocabulary.

norm_eps: f32

Epsilon added inside RMSNorm; defaults to 1e-5.

block_dim: Option<usize>

Hidden width used by LFM2 blocks, if set.

block_ff_dim: Option<usize>

Feed-forward width used by LFM2 blocks, if set.

layer_types: Vec<String>

Per-layer block type names (LFM2); empty when not given.

num_key_value_heads: Option<usize>

Number of key/value heads for grouped-query attention, if set.

rope_theta: Option<f64>

RoPE base frequency, if set.

quantization: Option<QuantizationConfig>

Quantization settings, present for quantized checkpoints.

bos_token_id: Option<u32>

Beginning-of-sequence token id, if set.

eos_token_id: Option<u32>

End-of-sequence token id, if set.

attention_dropout: Option<f32>

Attention dropout probability, if set.

model_type: Option<String>

Hugging Face model_type, e.g. "llama" or "lfm2".

tie_word_embeddings: Option<bool>

Whether lm_head shares the token-embedding matrix.

head_dim: Option<usize>

Per-head dimension when it is not hidden_size / num_attention_heads.