Struct ModelConfig 

pub struct ModelConfig {

    pub name: &'static str,
    pub n_layers: usize,
    pub hidden_dim: usize,
    pub n_heads: usize,
    pub n_kv_heads: usize,
    pub head_dim: usize,
    pub vocab_size: usize,
    pub rope_theta: f32,
    pub rms_norm_eps: f32,
    pub moe: MoeLayerConfig,
    pub attention: AttentionKind,
    pub sliding_window: Option<usize>,
    pub n_dense_leading_layers: usize,
    pub rope_freqs: Option<Vec<f32>>,
    pub rope_freqs_long: Option<Vec<f32>>,
    pub rope_freqs_short: Option<Vec<f32>>,
    pub rope_orig_ctx: Option<usize>,
    pub rope_dim: Option<usize>,
    pub rope_attn_factor: f32,
    pub rope_layout: RopeLayout,
    pub qk_norm_style: QkNormStyle,
    pub swa_pattern: Option<usize>,
    pub attn_logit_softcap: Option<f32>,
    pub final_logit_softcap: Option<f32>,
    pub embedding_scale: Option<f32>,
    pub attention_scale: Option<f32>,
    pub rope_theta_swa: Option<f32>,
    pub ffn_activation: FfnActivation,
    pub best_effort_fields: &'static [&'static str],
}

Fields

name: &'static str

n_layers: usize

hidden_dim: usize

n_heads: usize

n_kv_heads: usize

head_dim: usize

vocab_size: usize

rope_theta: f32

rms_norm_eps: f32

moe: MoeLayerConfig

attention: AttentionKind

Gqa for every preset except Kimi K3. Decoder’s forward pass does not yet branch on this – see AttentionKind’s doc comment.

sliding_window: Option<usize>

Mistral/Mixtral/Qwen2-family sliding-window attention: when set, every layer attends only to the most recent N cached positions instead of the full causal history (see ferrox_core::attention::causal_gqa_attention_windowed’s doc comment for the real source citations). None for every architecture that doesn’t use this (most models, including Qwen1.5/Qwen2-MoE’s real published config, which sets use_sliding_window: false despite carrying a sliding_window value – so this field being None/Some must come from that enable flag, not just the window-size field’s presence).

n_dense_leading_layers: usize

How many of the model’s first layers use an ordinary dense FFN (no expert routing at all) rather than the model’s MoE topology. Found by reading ik_llama.cpp’s real GGUF hparams-loading source (LLM_KV_LEADING_DENSE_BLOCK_COUNT): DeepSeek-2/3-family models don’t apply MoE uniformly to every layer – the first few layers are always dense. Zero means “every layer uses this model’s MoE topology,” the default for architectures that don’t do this.

rope_freqs: Option<Vec<f32>>

Llama 3/3.1/3.2’s real per-band RoPE frequency correction (the rope_freqs.weight GGUF tensor, head_dim/2 elements, TENSOR_NOT_REQUIRED so most architectures leave this None). See ferrox_core::attention::apply_rope_with_freq_factors’s doc comment for the real source and the real bug this closes: without it, every RoPE angle for a Llama-3-family checkpoint is computed slightly wrong, an error that compounds with position and eventually produces wrong logits (a spurious early EOS was the observed real symptom).

rope_freqs_long: Option<Vec<f32>>

LongRoPE’s two candidate factor sets, kept so the choice between them can be made when the run’s context size is known rather than at parse time. llama.cpp picks per request (llama_model::get_rope_factors reads cparams.n_ctx_seq), and the two sets are not interchangeable: Phi-4-mini’s short set is all ones (no correction at all) while its long set reaches 47. Choosing from the checkpoint’s advertised 131072 when the user runs at 4096 is a different model.

rope_freqs_short: Option<Vec<f32>>

rope_orig_ctx: Option<usize>

<arch>.rope.scaling.original_context_length — the threshold the choice above is made against.

rope_dim: Option<usize>

Rotary width when it is narrower than head_dim (<arch>.rope.dimension_count, llama.cpp hparams.n_rot). None means the whole head rotates, which is the common case. Phi-3/Phi-4 rotate 96 of 128.

rope_attn_factor: f32

LongRoPE/YaRN magnitude scaling (<arch>.rope.scaling.attn_factor, llama.cpp hparams.rope_attn_factor folded into cparams.yarn_attn_factor at llama-context.cpp:231, then applied as ggml rope_yarn’s mscale, which multiplies both cos and sin — so it scales the RoPE’d vector, at every position, whether or not any frequency correction is active.

Phi-4-mini ships 1.1902381. Ignoring it does not merely change long-context behaviour: q and k are both scaled, so every attention logit is off by attn_factor² and the softmax is sharper than the model’s. Measured symptom: ferrox and llama.cpp diverge from the eighth token of a greedy completion on the same GGUF.

1.0 for every architecture that does not set the key.

rope_layout: RopeLayout

RoPE pairing convention for this architecture – see RopeLayout. Independently of rope_freqs: a Llama checkpoint needs both Norm pairing and the per-band frequency factors.

qk_norm_style: QkNormStyle

How Q/K RMSNorm weights are applied when present (see crate::capability::QkNormStyle).

swa_pattern: Option<usize>

Gemma 2+/3 alternating SWA period. When Some(p), layer il uses sliding-window attention iff (il + 1) % p != 0 (llama.cpp set_swa_pattern convention); every p-th layer is full-attn.

attn_logit_softcap: Option<f32>

Attention logit soft-capping (Gemma 2+). Applied as softcap * tanh(score / softcap) before softmax.

final_logit_softcap: Option<f32>

Final logit soft-capping (Gemma 2+). Applied to lm_head output.

embedding_scale: Option<f32>

Input embedding scale (Gemma: sqrt(hidden_dim)).

attention_scale: Option<f32>

Optional override for the attention score scale baked into Q instead of the kernel’s default 1/sqrt(head_dim). When set, callers must pass score_scale = 1.0 into the attention kernel (llama.cpp Gemma: scale Q then build_attn(..., 1.0f)). Prefer leaving this None when the override equals 1/sqrt(head_dim).

rope_theta_swa: Option<f32>

RoPE base used on SWA layers (Gemma 3: defaults to 10000 when the GGUF omits rope.freq_base_swa; full-attn layers keep Self::rope_theta).

ffn_activation: FfnActivation

Dense/MoE FFN activation pairing.

best_effort_fields: &'static [&'static str]

Every field on this config that is a best-effort estimate rather than a confirmed value from an official config.json / GGUF file.

Implementations

impl ModelConfig

pub fn apply_runtime_context(&mut self, ctx: usize)

Re-picks the LongRoPE factor set now that the run’s context size is known, matching llama.cpp llama_model::get_rope_factors: rope_freqs.weight (Llama 3) always wins; otherwise the long set applies only when the context exceeds rope.scaling.original_context_length, and the short set otherwise.

A no-op for every checkpoint that ships neither set, which is all of them except the Phi-3/Phi-4 family today.

pub fn layer_is_dense(&self, layer_idx: usize) -> bool

True if layer layer_idx (0-indexed) should be built as an ordinary dense FFN rather than this model’s MoE topology.

pub fn layer_sliding_window(&self, layer_idx: usize) -> Option<usize>

Sliding-window size for layer il, honouring Gemma-style alternating SWA patterns. None means full causal attention.

pub fn layer_rope_theta(&self, layer_idx: usize) -> f32

RoPE frequency base for layer il (SWA layers may differ).

pub fn layer_attention_kind(&self, layer_idx: usize) -> LayerAttentionKind

Which attention mechanism layer layer_idx (0-indexed, ferrox’s usual convention) uses. For AttentionKind::Gqa every layer is LayerAttentionKind::Gqa; for AttentionKind::KimiHybrid, looks up layer_idx + 1 (the real kda_layers/full_attn_layers lists are 1-indexed – see KimiHybridAttention’s doc comment) in those real per-layer lists.

Panics

If layer_idx isn’t covered by either list of a KimiHybrid config – can’t happen for kimi_k3(), whose lists are tested (kimi_k3_hybrid_attention_layers_partition_every_layer_exactly_once) to partition every layer with no gaps, but a caller building a custom KimiHybridAttention must uphold the same invariant.

pub fn approx_active_params_per_token(&self) -> usize

Total parameter count implied by the MoE config, as a sanity check against the publicly reported total (this is an order of magnitude check, not an exact parameter-count reproduction).

impl ModelConfig

pub fn from_gguf(file: &impl TensorSource) -> Result<Self, LoadError>

Derives a ModelConfig from a real GGUF file’s own hyperparameter metadata, following llama.cpp’s general.architecture-prefixed key convention ({arch}.block_count, {arch}.embedding_length, {arch}.attention.head_count, {arch}.expert_count, …) rather than requiring a hand-written preset to already match the file’s shape exactly. This is what lets ferrox-server (and ferrox run-real) load an arbitrary checkpoint, not just the three hand-tuned presets in config.rs.

Fields with no corresponding metadata key fall back to widely-used llama.cpp defaults (documented inline) and are listed in the returned config’s best_effort_fields, following the same confirmed-vs-estimated discipline as the hand-written presets.

Trait Implementations

impl Clone for ModelConfig

fn clone(&self) -> ModelConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.