Struct ModelConfig 

pub struct ModelConfig {

    pub name: &'static str,
    pub n_layers: usize,
    pub hidden_dim: usize,
    pub n_heads: usize,
    pub n_kv_heads: usize,
    pub head_dim: usize,
    pub vocab_size: usize,
    pub rope_theta: f32,
    pub rms_norm_eps: f32,
    pub moe: MoeLayerConfig,
    pub attention: AttentionKind,
    pub sliding_window: Option<usize>,
    pub n_dense_leading_layers: usize,
    pub rope_freqs: Option<Vec<f32>>,
    pub rope_freqs_long: Option<Vec<f32>>,
    pub rope_freqs_short: Option<Vec<f32>>,
    pub rope_orig_ctx: Option<usize>,
    pub rope_dim: Option<usize>,
    pub rope_attn_factor: f32,
    pub rope_layout: RopeLayout,
    pub qk_norm_style: QkNormStyle,
    pub swa_pattern: Option<usize>,
    pub attn_logit_softcap: Option<f32>,
    pub final_logit_softcap: Option<f32>,
    pub embedding_scale: Option<f32>,
    pub attention_scale: Option<f32>,
    pub rope_theta_swa: Option<f32>,
    pub ffn_activation: FfnActivation,
    pub best_effort_fields: &'static [&'static str],
}

Fields

name: &'static str

n_layers: usize

hidden_dim: usize

n_heads: usize

n_kv_heads: usize

head_dim: usize

vocab_size: usize

rope_theta: f32

rms_norm_eps: f32

moe: MoeLayerConfig

attention: AttentionKind

Gqa for every preset except Kimi K3. Decoder’s forward pass does not yet branch on this – see AttentionKind’s doc comment.

sliding_window: Option<usize>

Mistral/Mixtral/Qwen2-family sliding-window attention: when set, every layer attends only to the most recent N cached positions instead of the full causal history (see ferrox_core::attention::causal_gqa_attention_windowed’s doc comment for the real source citations). None for every architecture that doesn’t use this (most models, including Qwen1.5/Qwen2-MoE’s real published config, which sets use_sliding_window: false despite carrying a sliding_window value – so this field being None/Some must come from that enable flag, not just the window-size field’s presence).

n_dense_leading_layers: usize

How many of the model’s first layers use an ordinary dense FFN (no expert routing at all) rather than the model’s MoE topology. Found by reading ik_llama.cpp’s real GGUF hparams-loading source (LLM_KV_LEADING_DENSE_BLOCK_COUNT): DeepSeek-2/3-family models don’t apply MoE uniformly to every layer – the first few layers are always dense. Zero means “every layer uses this model’s MoE topology,” the default for architectures that don’t do this.

rope_freqs: Option<Vec<f32>>

Llama 3/3.1/3.2’s real per-band RoPE frequency correction (the rope_freqs.weight GGUF tensor, head_dim/2 elements, TENSOR_NOT_REQUIRED so most architectures leave this None). See ferrox_core::attention::apply_rope_with_freq_factors’s doc comment for the real source and the real bug this closes: without it, every RoPE angle for a Llama-3-family checkpoint is computed slightly wrong, an error that compounds with position and eventually produces wrong logits (a spurious early EOS was the observed real symptom).

Not only a tensor: this is the resolved per-band divisor array, so a checkpoint declaring rope.scaling.type = "yarn" gets its YaRN frequency rewrite folded in here too (see ferrox_core::attention::yarn_freq_factors, and loader::yarn_scaling_from_gguf for what the file has to declare before that happens). A file carrying both a tensor and a YaRN declaration composes them by multiplication, as llama.cpp does (ggml_rope_cache_init divides by freq_factors and then runs rope_yarn). Consumers must therefore treat this as “the correction to apply”, not as “the tensor this file shipped”.

rope_freqs_long: Option<Vec<f32>>

LongRoPE’s two candidate factor sets, kept so the choice between them can be made when the run’s context size is known rather than at parse time. llama.cpp picks per request (llama_model::get_rope_factors reads cparams.n_ctx_seq), and the two sets are not interchangeable: Phi-4-mini’s short set is all ones (no correction at all) while its long set reaches 47. Choosing from the checkpoint’s advertised 131072 when the user runs at 4096 is a different model.

rope_freqs_short: Option<Vec<f32>>

rope_orig_ctx: Option<usize>

<arch>.rope.scaling.original_context_length — the threshold the choice above is made against.

rope_dim: Option<usize>

Rotary width when it is narrower than head_dim (<arch>.rope.dimension_count, llama.cpp hparams.n_rot). None means the whole head rotates, which is the common case. Phi-3/Phi-4 rotate 96 of 128.

rope_attn_factor: f32

LongRoPE/YaRN magnitude scaling (<arch>.rope.scaling.attn_factor, llama.cpp hparams.rope_attn_factor folded into cparams.yarn_attn_factor at llama-context.cpp:231, then applied as ggml rope_yarn’s mscale, which multiplies both cos and sin — so it scales the RoPE’d vector, at every position, whether or not any frequency correction is active.

Phi-4-mini ships 1.1902381. Ignoring it does not merely change long-context behaviour: q and k are both scaled, so every attention logit is off by attn_factor² and the softmax is sharper than the model’s. Measured symptom: ferrox and llama.cpp diverge from the eighth token of a greedy completion on the same GGUF.

1.0 for every architecture that does not set the key.

rope_layout: RopeLayout

RoPE pairing convention for this architecture – see RopeLayout. Independently of rope_freqs: a Llama checkpoint needs both Norm pairing and the per-band frequency factors.

qk_norm_style: QkNormStyle

How Q/K RMSNorm weights are applied when present (see crate::capability::QkNormStyle).

swa_pattern: Option<usize>

Gemma 2+/3 alternating SWA period. When Some(p), layer il uses sliding-window attention iff (il + 1) % p != 0 (llama.cpp set_swa_pattern convention); every p-th layer is full-attn.

attn_logit_softcap: Option<f32>

Attention logit soft-capping (Gemma 2+). Applied as softcap * tanh(score / softcap) before softmax.

final_logit_softcap: Option<f32>

Final logit soft-capping (Gemma 2+). Applied to lm_head output.

embedding_scale: Option<f32>

Input embedding scale (Gemma: sqrt(hidden_dim)).

attention_scale: Option<f32>

Optional override for the attention score scale baked into Q instead of the kernel’s default 1/sqrt(head_dim). When set, callers must pass score_scale = 1.0 into the attention kernel (llama.cpp Gemma: scale Q then build_attn(..., 1.0f)). Prefer leaving this None when the override equals 1/sqrt(head_dim).

rope_theta_swa: Option<f32>

RoPE base used on SWA layers (Gemma 3: defaults to 10000 when the GGUF omits rope.freq_base_swa; full-attn layers keep Self::rope_theta).

ffn_activation: FfnActivation

Dense/MoE FFN activation pairing.

best_effort_fields: &'static [&'static str]

Every field on this config that is a best-effort estimate rather than a confirmed value from an official config.json / GGUF file.

Implementations

impl ModelConfig

pub fn apply_runtime_context(&mut self, ctx: usize)

Re-picks the LongRoPE factor set now that the run’s context size is known, matching llama.cpp llama_model::get_rope_factors: rope_freqs.weight (Llama 3) always wins; otherwise the long set applies only when the context exceeds rope.scaling.original_context_length, and the short set otherwise.

A no-op for every checkpoint that ships neither set, which is all of them except the Phi-3/Phi-4 family today.

Because it re-picks rope_freqs wholesale it would also discard a YaRN rewrite folded into that field at parse time (see Self::rope_freqs). No real checkpoint hits that: LongRoPE files declare rope.scaling.type = "longrope", which the loader’s YaRN arm deliberately does not claim, so the two never populate the field on the same file.

pub fn layer_is_dense(&self, layer_idx: usize) -> bool

True if layer layer_idx (0-indexed) should be built as an ordinary dense FFN rather than this model’s MoE topology.

pub fn layer_sliding_window(&self, layer_idx: usize) -> Option<usize>

Sliding-window size for layer il, honouring Gemma-style alternating SWA patterns. None means full causal attention.

pub fn kv_block_window(&self) -> Option<usize>

The narrowest sliding window any layer of this model uses, or None if every layer is full-causal.

For an alternating-SWA model (gpt-oss, Gemma-3) the full-attention layers impose no constraint on the KV block layout and the sliding ones impose the window – so the model’s constraint is simply the window, present as soon as any layer slides. A model that is 5/6 full-attention is not 5/6 exempt: one mis-aligned sliding layer corrupts the answer.

pub fn uniform_sliding_window(&self) -> Option<usize>

The window EVERY layer slides by, or None if any layer attends over the whole history.

This is the opposite question to Self::kv_block_window, and the difference is the whole reason both exist. That one asks “does any layer constrain the block layout”, so one sliding layer is enough. This one asks “may a page that has fallen behind the window be taken away”, and there one full-attention layer is enough to say no.

A page group holds one block in every layer and is freed as a unit, so on an alternating-SWA model (gpt-oss, Gemma-3) freeing the group behind the window would take the full-attention layers’ block with it – and those layers still read position 0 at every step. The result is not a crash: the block is reused by another request and the full layers attend over its bytes. So this returns None for the alternating case, and a mixed-window model (were one to appear) gets None too rather than the narrowest window, because the widest is the one that must still be readable.

pub fn kv_block_layout(&self, desired_block_size: usize) -> BlockLayout

The KV cache block layout to use for this model, given the block size an operator asked for.

The requested size is rounded down to something that divides the window (see ferrox_core::kv_swa), so a config that would straddle the window boundary becomes a smaller block rather than a startup failure or – much worse – a silently wrong mask.

pub fn layer_rope_theta(&self, layer_idx: usize) -> f32

RoPE frequency base for layer il (SWA layers may differ).

pub fn layer_attention_kind(&self, layer_idx: usize) -> LayerAttentionKind

Which attention mechanism layer layer_idx (0-indexed, ferrox’s usual convention) uses. For AttentionKind::Gqa every layer is LayerAttentionKind::Gqa; for AttentionKind::KimiHybrid, looks up layer_idx + 1 (the real kda_layers/full_attn_layers lists are 1-indexed – see KimiHybridAttention’s doc comment) in those real per-layer lists.

Panics

If layer_idx isn’t covered by either list of a KimiHybrid config – can’t happen for kimi_k3(), whose lists are tested (kimi_k3_hybrid_attention_layers_partition_every_layer_exactly_once) to partition every layer with no gaps, but a caller building a custom KimiHybridAttention must uphold the same invariant.

pub fn approx_active_params_per_token(&self) -> usize

Total parameter count implied by the MoE config, as a sanity check against the publicly reported total (this is an order of magnitude check, not an exact parameter-count reproduction).

impl ModelConfig

pub fn from_gguf(file: &impl TensorSource) -> Result<Self, LoadError>

Derives a ModelConfig from a real GGUF file’s own hyperparameter metadata, following llama.cpp’s general.architecture-prefixed key convention ({arch}.block_count, {arch}.embedding_length, {arch}.attention.head_count, {arch}.expert_count, …) rather than requiring a hand-written preset to already match the file’s shape exactly. This is what lets ferrox-server (and ferrox run-real) load an arbitrary checkpoint, not just the three hand-tuned presets in config.rs.

Fields with no corresponding metadata key fall back to widely-used llama.cpp defaults (documented inline) and are listed in the returned config’s best_effort_fields, following the same confirmed-vs-estimated discipline as the hand-written presets.

Trait Implementations

impl Clone for ModelConfig

fn clone(&self) -> ModelConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.