Struct ModelConfig 

pub struct ModelConfig {

    pub name: &'static str,
    pub n_layers: usize,
    pub n_mtp_blocks: usize,
    pub hidden_dim: usize,
    pub n_heads: usize,
    pub n_kv_heads: usize,
    pub head_dim: usize,
    pub v_head_dim: Option<usize>,
    pub vocab_size: usize,
    pub rope_theta: f32,
    pub rms_norm_eps: f32,
    pub post_norm_eps: f32,
    pub norm_function: NormFunction,
    pub moe: MoeLayerConfig,
    pub attention: AttentionKind,
    pub sliding_window: Option<usize>,
    pub n_dense_leading_layers: usize,
    pub moe_interleave_step: Option<usize>,
    pub rope_freqs: Option<RopeFreqs>,
    pub rope_freqs_long: Option<Vec<f32>>,
    pub rope_freqs_short: Option<Vec<f32>>,
    pub rope_orig_ctx: Option<usize>,
    pub rope_dim: Option<usize>,
    pub rope_dim_swa: Option<usize>,
    pub rope_attn_factor: f32,
    pub rope_layout: RopeLayout,
    pub qk_norm_style: QkNormStyle,
    pub swa_layers: SwaLayers,
    pub rope_layers: RopeLayers,
    pub layer_shapes: LayerShapes,
    pub attn_logit_softcap: Option<f32>,
    pub final_logit_softcap: Option<f32>,
    pub embedding_scale: Option<f32>,
    pub residual_scale: Option<f32>,
    pub normed_residual_scale: Option<f32>,
    pub logit_multiplier: Option<f32>,
    pub attention_scale: Option<f32>,
    pub clamp_kqv: Option<f32>,
    pub attn_temperature: Option<AttnTemperature>,
    pub router_input: RouterInput,
    pub block_sub_norms: bool,
    pub parallel_residual: bool,
    pub learned_positions: bool,
    pub attn_value_scale: Option<f32>,
    pub alibi_max_bias: Option<f32>,
    pub layer_loops: Option<LayerLoops>,
    pub skip_stream: bool,
    pub parallel_ssm: bool,
    pub swa_chunked: bool,
    pub weightless_qk_norm: bool,
    pub rope_theta_swa: Option<f32>,
    pub ffn_activation: FfnActivation,
    pub best_effort_fields: &'static [&'static str],
}

Fields

name: &'static str

n_layers: usize

Decoder layers: llama.cpp’s n_layer(), which is the file’s block_count MINUS Self::n_mtp_blocks.

n_mtp_blocks: usize

NextN / MTP blocks the file appends after the trunk, inside its block_count, which llama.cpp creates TENSOR_SKIP and never runs (crate::mtp_blocks). Their tensors are blk.N.* for n_layers <= N < n_layers + n_mtp_blocks; the loader marks them deliberately unread. Zero for every architecture whose graph does not read nextn_predict_layers.

hidden_dim: usize

n_heads: usize

Query heads of the WIDEST layer. Every layer’s for a uniform model, which is every model but the per-layer-shape ones (crate::layer_shapes); a layer body must read its own count through Self::layer_shape, never this field.

n_kv_heads: usize

KV heads of the WIDEST layer, so that a budget priced from it over-counts rather than under-counts a heterogeneous model. Same rule as n_heads: per-layer computation reads Self::layer_shape; caches come from Self::new_kv_caches.

head_dim: usize

The K head width (attention.key_length, llama.cpp n_embd_head_k): the width of every Q and K head, the width RoPE rotates within, and the 1/sqrt of the attention scale.

v_head_dim: Option<usize>

The V head width (attention.value_length, n_embd_head_v) WHEN IT DIFFERS from Self::head_dim; None means V heads are K’s width, which is every architecture but MiMo-V2 (head_dim: 192, v_head_dim: 128). Read it through Self::v_head_dim, never here: an Option rather than a second usize so that a config whose head_dim is set or changed cannot leave a stale V width beside it – the two-fields-that-must-agree shape. See crate::kv_head_dims for which architectures may declare them apart and which fused paths refuse when they are.

vocab_size: usize

rope_theta: f32

rms_norm_eps: f32

post_norm_eps: f32

The epsilon the POST-attention and POST-FFN norms run at.

Equal to Self::rms_norm_eps for every architecture but the one whose graph writes a literal (crate::norm:: POST_NORM_EPS_LITERAL, muse-glimmer.cpp:63). Set by the loader from that table so the two cannot be given different answers by two callers, and read through Self::post_norm_eps.

norm_function: NormFunction

The norm FUNCTION every weighted site applies (crate::norm::norm_function): the architecture’s, or, for crate::norm::NORM_BY_RMS_EPS_KEY, the file’s.

moe: MoeLayerConfig

attention: AttentionKind

Gqa for every preset except Kimi K3. Decoder’s forward pass does not yet branch on this – see AttentionKind’s doc comment.

sliding_window: Option<usize>

Mistral/Mixtral/Qwen2-family sliding-window attention: when set, every layer attends only to the most recent N cached positions instead of the full causal history (see frink_core::attention::causal_gqa_attention_windowed’s doc comment for the real source citations). None for every architecture that doesn’t use this (most models, including Qwen1.5/Qwen2-MoE’s real published config, which sets use_sliding_window: false despite carrying a sliding_window value – so this field being None/Some must come from that enable flag, not just the window-size field’s presence).

n_dense_leading_layers: usize

How many of the model’s first layers use an ordinary dense FFN (no expert routing at all) rather than the model’s MoE topology. Found by reading ik_llama.cpp’s real GGUF hparams-loading source (LLM_KV_LEADING_DENSE_BLOCK_COUNT): DeepSeek-2/3-family models don’t apply MoE uniformly to every layer – the first few layers are always dense. Zero means “every layer uses this model’s MoE topology,” the default for architectures that don’t do this.

moe_interleave_step: Option<usize>

{arch}.interleave_moe_layer_step where the loader honours it (crate::moe_interleave::INTERLEAVE_STEP_HONOURED_BY_LOADER): layer il is MoE when (il + 1) % step == 0. None everywhere else, including the ERNIE files whose step is 1.

rope_freqs: Option<RopeFreqs>

Llama 3/3.1/3.2’s real per-band RoPE frequency correction (the rope_freqs.weight GGUF tensor, head_dim/2 elements, TENSOR_NOT_REQUIRED so most architectures leave this None). See frink_core::attention::apply_rope_with_freq_factors’s doc comment for the real source and the real bug this closes: without it, every RoPE angle for a Llama-3-family checkpoint is computed slightly wrong, an error that compounds with position and eventually produces wrong logits (a spurious early EOS was the observed real symptom).

Not only a tensor: this is the resolved per-band divisor array, so a checkpoint declaring rope.scaling.type = "yarn" gets its YaRN frequency rewrite folded in here too (see frink_core::attention::yarn_freq_factors, and loader::yarn_scaling_from_gguf for what the file has to declare before that happens). A file carrying both a tensor and a YaRN declaration composes them by multiplication, as llama.cpp does (ggml_rope_cache_init divides by freq_factors and then runs rope_yarn). Consumers must therefore treat this as “the correction to apply”, not as “the tensor this file shipped”.

Per-LAYER, because llama.cpp’s is: see RopeFreqs. Read it through Self::layer_rope, never field-by-field.

rope_freqs_long: Option<Vec<f32>>

LongRoPE’s two candidate factor sets, kept so the choice between them can be made when the run’s context size is known rather than at parse time. llama.cpp picks per request (llama_model::get_rope_factors reads cparams.n_ctx_seq), and the two sets are not interchangeable: Phi-4-mini’s short set is all ones (no correction at all) while its long set reaches 47. Choosing from the checkpoint’s advertised 131072 when the user runs at 4096 is a different model.

rope_freqs_short: Option<Vec<f32>>

rope_orig_ctx: Option<usize>

<arch>.rope.scaling.original_context_length — the threshold the choice above is made against.

rope_dim: Option<usize>

Rotary width when it is narrower than head_dim (<arch>.rope.dimension_count, llama.cpp hparams.n_rot). None means the whole head rotates, which is the common case. Phi-3/Phi-4 rotate 96 of 128.

This is the FULL-attention layers’ width, llama.cpp’s n_rot_full; Self::rope_dim_swa is the sliding layers’.

rope_dim_swa: Option<usize>

The SLIDING layers’ rotary width when it differs from Self::rope_dim – llama.cpp’s n_rot_swa, read from rope.dimension_count_swa or halved-from-full for step35 (crate::swa_geometry), consumed through n_rot(il) (llama-hparams.cpp:85-91). None means the sliding layers rotate the same width as the full ones, which is every architecture but the ones the table names. Some(head_dim) is the whole head, and Self::layer_rope normalises it.

rope_attn_factor: f32

LongRoPE/YaRN magnitude scaling (<arch>.rope.scaling.attn_factor, llama.cpp hparams.rope_attn_factor folded into cparams.yarn_attn_factor at llama-context.cpp:231, then applied as ggml rope_yarn’s mscale, which multiplies both cos and sin — so it scales the RoPE’d vector, at every position, whether or not any frequency correction is active.

Phi-4-mini ships 1.1902381. Ignoring it does not merely change long-context behaviour: q and k are both scaled, so every attention logit is off by attn_factor² and the softmax is sharper than the model’s. Measured symptom: frink and llama.cpp diverge from the eighth token of a greedy completion on the same GGUF.

1.0 for every architecture that does not set the key.

rope_layout: RopeLayout

RoPE pairing convention for this architecture – see RopeLayout. Independently of rope_freqs: a Llama checkpoint needs both Norm pairing and the per-band frequency factors.

qk_norm_style: QkNormStyle

How Q/K RMSNorm weights are applied when present (see crate::capability::QkNormStyle).

swa_layers: SwaLayers

WHICH LAYERS SLIDE – llama.cpp’s is_swa_impl[il], as a period with a phase, the file’s own per-layer array, or every layer. See crate::swa_layers. Meaningless without Self::sliding_window; Self::layer_sliding_window is the one accessor that combines the two.

Getting the phase wrong is not a near miss: on a 32-layer period-4 model the two phases disagree about SIXTEEN layers, each of which then attends over the wrong span at full speed. capability::default_swa_layout carries the per-arch value, transcribed from llama.cpp.

rope_layers: RopeLayers

WHICH LAYERS ROTATE – llama.cpp’s per-layer use_rope.

crate::rope_layers::RopeLayers::All for every architecture that writes no gate, which is 134 of llama.cpp’s 140. The rule and the table that assigns it live in crate::rope_layers; nothing else in this crate may branch on an architecture name to decide it, and Self::layer_rope returning None is the only way a call site learns of it.

layer_shapes: LayerShapes

WHICH LAYERS HAVE WHICH SHAPE – llama.cpp’s n_head(il), n_head_kv(il) and n_ff(il).

Uniform for every architecture whose graph reads layer 0, which is all but the rows in layer_shapes::PER_LAYER_SHAPE_ARCHS. Self::layer_shape is the one accessor; the fused Metal launches and the CUDA resident KV are fenced off any model that is not Uniform, because each holds one geometry.

attn_logit_softcap: Option<f32>

Attention logit soft-capping (Gemma 2+). Applied as softcap * tanh(score / softcap) before softmax.

final_logit_softcap: Option<f32>

Final logit soft-capping (Gemma 2+). Applied to lm_head output.

embedding_scale: Option<f32>

Input embedding scale (Gemma: sqrt(hidden_dim); Granite: {arch}.embedding_scale).

residual_scale: Option<f32>

Multiplier applied to EVERY branch output – attention and FFN alike – immediately before it rejoins the residual stream (Granite residual_multiplier, src/models/granite.cpp:235-238 and :288-292).

None means the plain hidden += branch every other architecture computes. The decoder never applies this field itself: crate::scalar_multipliers::residual_add is the one residual add, and it takes this value as a parameter, because decoder.rs spells the add out eighteen times and eighteen hand-written copies that must agree about one scalar is the defect shape this repo keeps paying for.

normed_residual_scale: Option<f32>

Some(s): each sublayer’s PRE-NORM OUTPUT, times s, REPLACES the residual stream its branch joins, and the layer input is discarded (crate::normed_residual; minimax-01.cpp:249,428).

Never Some together with Self::residual_scale – one column of MultiplierSupport resolves both – and Some(1.0) is a real value here, because the field carries the topology as well as the multiplier.

logit_multiplier: Option<f32>

Multiplier applied to the lm_head’s output, after the projection and before Self::final_logit_softcap.

Already resolved into a MULTIPLIER at load time, whichever direction the architecture’s graph states it in: Granite divides by {arch}.logit_scale (granite.cpp:180), so this field holds 1.0 / logit_scale. Keeping the direction in crate::scalar_multipliers rather than here is what lets the decoder have exactly one multiply, and stops a second architecture with the opposite convention from needing a second field.

Guaranteed positive when Some, and that is load-bearing rather than incidental: a Metal decode stack may fold the lm_head and return an argmax token id, which is only sound while every post-head transform is monotone increasing.

attention_scale: Option<f32>

Optional override for the attention score scale baked into Q instead of the kernel’s default 1/sqrt(head_dim). When set, callers must pass score_scale = 1.0 into the attention kernel (llama.cpp Gemma: scale Q then build_attn(..., 1.0f)). Prefer leaving this None when the override equals 1/sqrt(head_dim).

clamp_kqv: Option<f32>

Symmetric clamp on the Q, K and V projections ({arch}.attention.clamp_kqv), applied after the QKV bias and before the QK-norm and RoPE – llama.cpp’s build_qkv (llama-graph.cpp:1611-1652).

Some(c) only when the architecture’s graph clamps AND the file declares a positive value; llama.cpp’s own test is > 0.0f, so zero and a negative value are “no clamp” and resolve to None here rather than to a clamp that zeroes every projection. The resolution lives in crate::clamp_kqv; the decoder applies it through ONE helper shared by every host body, and the fused Metal launches are fenced off by Decoder::metal_can_serve_model because no kernel implements it.

attn_temperature: Option<AttnTemperature>

Per-position attention temperature – llama.cpp’s llm_graph_input_attn_temp, the [n_tokens] vector log(floor((pos + offset) / floor_scale) + 1) * scale + 1 that mistral3.cpp:153-156 multiplies into Q after RoPE, before build_attn, with kq_scale untouched. See crate::attn_temperature for the census (three graphs of 155) and the resolution.

Some only for an architecture whose graph builds the input AND a file declaring a nonzero attention.temperature_scale; the key on any other architecture is dead metadata upstream and is ignored here the same way. Applied through ONE helper, Decoder::apply_attn_temperature, on every host body, and fenced off the fused Metal launches by Decoder::metal_can_serve_model, because none has a per-token Q scale uniform.

router_input: RouterInput

WHICH TENSOR THE MoE ROUTER READS – the normed FFN input for every graph but one, the raw layer input for smallthinker (smallthinker.cpp:111). See crate::router_input for the census (four graphs of 155 pass a precomputed probs_in, one on the generic path) and the seam. Decoder::router_operand is the ONE place the operand is captured, and the GPU router paths refuse a model whose operand they cannot read (Decoder::gpu_router_matches_host_routing).

block_sub_norms: bool

Whether this model’s blocks norm INSIDE the two sublayers: BitNet’s attn_sub_norm (on the attention output, BEFORE wo) and ffn_sub_norm (on silu(gate) * up, BEFORE down), bitnet.cpp:24,36,101-106,135-140. See crate::sub_norms for the census (one graph of 155) and the two readers: the loader, which REQUIRES the pair when this is set, and Decoder::metal_can_serve_model, which refuses every fused launch, since none has a norm at either site.

parallel_residual: bool

Whether any layer of this model is a PARALLEL residual, x + attn(norm(x)) + ffn(norm(x)) (crate::parallel_residual; gptneox under its key, plamo always, stablelm per layer by tensor presence). The per-layer fact is MoeWeights::parallel; this is the model-level one Decoder::metal_can_serve_model reads, because every fused Metal launch bakes the pre-FFN norm over the post-attention residual into its kernel.

learned_positions: bool

Whether this model adds a learned position table to its token embeddings (crate::position_embd; gpt2, starcoder). The table itself is Decoder::position_embd; this is the model-level fact Decoder::metal_can_serve_model reads, because the GPU embedding gather has no add and the fused stacks never see pos.

attn_value_scale: Option<f32>

{arch}.attention.value_scale: MiMo-V2 multiplies the attention branch by it AFTER wo (mimo2.cpp:180-183; every real export carries 0.707). None for no scale; see crate::attn_value_scale for the one reader and the values that mean none. Applied in Decoder::attn_out_to_residual_rows; the fused Metal launches refuse a model that has one.

alibi_max_bias: Option<f32>

llama.cpp’s f_max_alibi_bias when it is positive: the model positions by ALiBi and rotates nothing (crate::alibi for which graphs and where each gets the number; frink_core::alibi for the per-head slopes, which Decoder::alibi_slopes holds). None for every other model. The fused Metal launches and the CUDA resident attention refuse a model that has one: their kernels add no per-key bias.

layer_loops: Option<LayerLoops>

Nanbeige’s num_loops: Some when the model’s logical layers are several passes over its physical ones (nanbeige.cpp:19-31). Self::n_layers is then the LOGICAL count, Decoder::layers stays physical, and Decoder::layer_for maps one to the other. See crate::layer_loops; the fused Metal launches refuse a looped model.

skip_stream: bool

Talkie’s embedding skip stream (talkie.cpp:50-52,123-126): the embeddings are RMS-normed without a weight before layer 0 and every layer adds that vector, times its own layer_output_scale, after its FFN residual. See crate::skip_stream; the fused Metal launches refuse a model that has one.

parallel_ssm: bool

Every attention layer ALSO runs a Mamba-2 block on the same normed input, the two outputs summed (falcon-h1.cpp:137-161; crate::mamba2::PARALLEL_WITH_ATTENTION). The layer’s cache holds the attention rows AND the block’s RecurrentState, so Self::has_recurrent_layers is true and the fused Metal launches refuse the model.

swa_chunked: bool

The sliding layers’ window is a CHUNK (crate::chunked_swa): a query sees its own sliding_window-sized chunk and nothing before it. Self::layer_window_for_query is the per-query window the single-query kernels take for it; the fused Metal launches refuse the model.

weightless_qk_norm: bool

A per-head RMSNorm with no weight on Q and K after RoPE, on the layers that rotate (crate::weightless_qk_norm, Llama 4’s Llama4TextL2Norm). Applied at the post-RoPE QK-norm hook; the fused Metal launches refuse the model.

rope_theta_swa: Option<f32>

RoPE base used on SWA layers (Gemma 3: defaults to 10000 when the GGUF omits rope.freq_base_swa; full-attn layers keep Self::rope_theta).

ffn_activation: FfnActivation

Dense/MoE FFN activation pairing.

best_effort_fields: &'static [&'static str]

Every field on this config that is a best-effort estimate rather than a confirmed value from an official config.json / GGUF file.

Implementations

impl ModelConfig

pub fn layer_ffn_acts(&self, il: usize) -> LayerFfnActs

Layer il’s FFN activations, with their parameters, by site. THE accessor: every FFN body – routed, shared, dense, batched, slotted – reads its activation here and nowhere else.

For every architecture but the parameterised ones this is the same answer for every il and both sites, which is what ffn_activation used to be converted to directly; that conversion no longer exists, because it could not be written for a variant that needs the layer.

pub fn model_ffn_act(&self) -> Option<GluAct>

The ONE activation every layer of this model runs, or None when it varies by layer – the whole-model question the fused Metal stacks and their eligibility checks ask, since each takes one activation uniform for a whole run of layers.

None is a refusal at every such site. It is not derived by comparing layer_ffn_act across layers, because a parameterised activation is per layer BY TYPE: a two-layer xIELU model whose two parameter sets happen to be equal is still not something a kernel with no xIELU in it can serve.

pub fn ffn_is_ungated(&self) -> bool

Does this model’s FFN have no gate matrix on disk?

The two ungated activations share the loader’s aliasing (load_dense_expert), so the question is asked once here rather than as == ReluSqr at the site, where the second variant would have been forgotten.

impl ModelConfig

pub fn v_head_dim(&self) -> usize

The V head width: the width of every V head, of each head’s attention output, and so of o_proj’s input (n_heads * v_head_dim()). Self::head_dim unless the file declared attention.value_length apart from attention.key_length on an architecture that sizes them apart (crate::kv_head_dims).

pub fn kv_head_dims_split(&self) -> bool

Whether V heads are a different width from K heads. The fact every fused path refuses on.

pub fn apply_runtime_context(&mut self, ctx: usize)

Re-picks the LongRoPE factor set now that the run’s context size is known, matching llama.cpp llama_model::get_rope_factors: rope_freqs.weight (Llama 3) always wins; otherwise the long set applies only when the context exceeds rope.scaling.original_context_length, and the short set otherwise.

A no-op for every checkpoint that ships neither set, which is all of them except the Phi-3/Phi-4 family today.

Because it re-picks rope_freqs wholesale it would also discard a YaRN rewrite folded into that field at parse time (see Self::rope_freqs). No real checkpoint hits that: LongRoPE files declare rope.scaling.type = "longrope", which the loader’s YaRN arm deliberately does not claim, so the two never populate the field on the same file. The same caveat now covers RopeFreqs::swa, and for the same reason: no LongRoPE checkpoint has alternating SWA layers.

pub fn layer_is_dense(&self, layer_idx: usize) -> bool

True if layer layer_idx (0-indexed) should be built as an ordinary dense FFN rather than this model’s MoE topology: the leading-dense prefix, and, where the loader honours the interleave step (crate::moe_interleave, llama4.cpp:64), a layer with (il + 1) % step != 0.

pub fn layer_sliding_window(&self, layer_idx: usize) -> Option<usize>

Sliding-window size for layer il, honouring Gemma-style alternating SWA patterns. None means full causal attention.

pub fn layer_window_for_query(&self, il: usize, pos: usize) -> Option<usize>

The window the single-query kernels take for a query at pos on layer il: the layer’s sliding window, or, when the window is chunked (crate::chunked_swa), the pos % chunk + 1 positions of the query’s own chunk. None for a full layer.

pub fn batch_window( &self, il: usize, start_pos: usize, batch_size: usize, ) -> BatchWindow

The window a batch of batch_size queries starting at start_pos takes on layer il, for the batched prefill body: one window for the blocked kernel, or one per query where a chunked layer’s queries do not share a chunk start.

pub fn kv_block_window(&self) -> Option<usize>

The narrowest sliding window any layer of this model uses, or None if every layer is full-causal.

For an alternating-SWA model (gpt-oss, Gemma-3) the full-attention layers impose no constraint on the KV block layout and the sliding ones impose the window – so the model’s constraint is simply the window, present as soon as any layer slides. A model that is 5/6 full-attention is not 5/6 exempt: one mis-aligned sliding layer corrupts the answer.

pub fn uniform_sliding_window(&self) -> Option<usize>

The window EVERY layer slides by, or None if any layer attends over the whole history.

This is the opposite question to Self::kv_block_window, and the difference is the whole reason both exist. That one asks “does any layer constrain the block layout”, so one sliding layer is enough. This one asks “may a page that has fallen behind the window be taken away”, and there one full-attention layer is enough to say no.

A page group holds one block in every layer and is freed as a unit, so on an alternating-SWA model (gpt-oss, Gemma-3) freeing the group behind the window would take the full-attention layers’ block with it – and those layers still read position 0 at every step. The result is not a crash: the block is reused by another request and the full layers attend over its bytes. So this returns None for the alternating case, and a mixed-window model (were one to appear) gets None too rather than the narrowest window, because the widest is the one that must still be readable.

pub fn kv_block_layout(&self, desired_block_size: usize) -> BlockLayout

The KV cache block layout to use for this model, given the block size an operator asked for.

The requested size is rounded down to something that divides the window (see frink_core::kv_swa), so a config that would straddle the window boundary becomes a smaller block rather than a startup failure or – much worse – a silently wrong mask.

pub fn post_norm_eps(&self) -> f32

ALL THREE halves of layer il’s RoPE: the frequency base, the per-band divisors, which llama.cpp varies per layer together (llama-model.cpp:2029-2035, and see RopeFreqs), and the rotary WIDTH, which it varies by the same sliding-or-full fact (n_rot(il), llama-hparams.cpp:85-91; Self::rope_dim_swa).

Every RoPE call site takes the pair from here. Splitting them was the defect: layer_rope_theta varied the base per layer while rope_freqs was one global vector, so Gemma-3 4B/12B/27B roped their sliding layers at scaled positions llama.cpp leaves unscaled.

None means this layer does not rotate at all, which is llama.cpp’s per-layer use_rope gate – crate::rope_layers holds the rule and the six architectures that have one. It is an Option rather than a separate predicate beside the pair precisely so that a call site cannot take the base and the divisors without also answering “does this layer rotate”: that is the third thing the three had to agree about, and two of them were already one value for this reason. The epsilon the post-attention / post-FFN norms run at. One accessor so a site cannot read the model’s epsilon by habit.

pub fn layer_rope(&self, layer_idx: usize) -> Option<LayerRopeParams<'_>>

pub fn rope_dim_varies_by_layer(&self) -> bool

True when the sliding layers rotate a different width from the full ones – the whole-model fact the fused Metal launches refuse on, since each takes ONE rot_dim uniform for every layer.

Derived from Self::layer_rope rather than from the field, so a rope_dim_swa that merely restates rope_dim (or the whole head) is not a difference.

pub fn layer_rotates(&self, layer_idx: usize) -> bool

Does layer il rotate at all? Derived from Self::layer_rope rather than restated beside it, so the two can never disagree.

pub fn any_layer_unrotated(&self) -> bool

True when at least one layer of this model gets no rotation – the whole-model question, for the eligibility checks and the receipts that want it once rather than per layer.

pub fn layer_rope_theta(&self, layer_idx: usize) -> Option<f32>

RoPE frequency base for layer il (SWA layers may differ), or None where the layer does not rotate.

Prefer Self::layer_rope anywhere the divisors are needed too, which is every site that actually rotates something. This one is for the callers that only report or compare the base.

pub fn rope_freqs_vary_by_layer(&self) -> bool

True when the sliding layers need different per-band divisors from the full-attention ones, i.e. when one freq_factors slice cannot describe every layer of this model. Gemma-3 4B/12B/27B are the shape that answers yes.

It is NOT an eligibility check any more. It was one: the fused Metal stacks took a single slice for a whole run of layers and refused a model that answered yes here. They now take a frink_metal::attn::LayerRope per layer, so this is a statement about the checkpoint and nothing else – which is all the loader tests ever wanted from it.

pub fn layer_attention_kind(&self, layer_idx: usize) -> LayerAttentionKind

Which attention mechanism layer layer_idx (0-indexed, frink’s usual convention) uses. For AttentionKind::Gqa every layer is LayerAttentionKind::Gqa; for AttentionKind::KimiHybrid, looks up layer_idx + 1 (the real kda_layers/full_attn_layers lists are 1-indexed – see KimiHybridAttention’s doc comment) in those real per-layer lists.

Panics

If layer_idx isn’t covered by either list of a KimiHybrid config – can’t happen for kimi_k3(), whose lists are tested (kimi_k3_hybrid_attention_layers_partition_every_layer_exactly_once) to partition every layer with no gaps, but a caller building a custom KimiHybridAttention must uphold the same invariant.

pub fn approx_active_params_per_token(&self) -> usize

Total parameter count implied by the MoE config, as a sanity check against the publicly reported total (this is an order of magnitude check, not an exact parameter-count reproduction).

impl ModelConfig

pub fn has_recurrent_layers(&self) -> bool

True when any layer carries a RecurrentState between tokens (AttnShape::is_recurrent): the fact every caller that rolls a cache back to a middle position – speculative verification, the draft model, the prefix cache – is fenced on (frink_core::recurrent_state).

pub fn layer_cache_geometry(&self, il: usize) -> (usize, usize, usize)

Layer il’s cache geometry (AttnShape::cache_geometry at this model’s widths).

pub fn layer_shape(&self, il: usize) -> LayerShape

Layer il’s shape. THE accessor: every layer body reads its head counts here and nowhere else.

pub fn new_kv_caches(&self) -> Vec<KvCache>

One contiguous cache per layer, each sized for that layer.

The twenty-odd call sites that used to spell KvCache::new(config.n_kv_heads, config.head_dim) per layer were twenty copies of one geometry decision, and every one of them was wrong for a model whose layers differ.

pub fn new_kv_caches_with_capacity(&self, max_seq_len: usize) -> Vec<KvCache>

The same, pre-allocated for max_seq_len positions.

pub fn new_kv_caches_with_pool( &self, pool: &Arc<Mutex<KvBlockPool>>, max_seq_len: usize, ) -> Result<Vec<KvCache>, KvPoolExhausted>

The same, each layer’s storage leased from pool. The first layer that cannot be leased fails the whole set, as before.

pub fn new_paged_kv( &self, block_size: usize, blocks_per_layer: usize, ) -> SharedPagedKv

One paged store per layer, each sized for that layer.

pub fn kv_heads_all_layers(&self) -> usize

KV heads summed over every layer: what a per-token memory budget multiplies by head_dim * elem_size. n_layers * n_kv_heads for a uniform model, and an over-count for a heterogeneous one wherever it is still spelled that way.

impl ModelConfig

pub fn from_gguf(file: &impl TensorSource) -> Result<Self, LoadError>

Derives a ModelConfig from a real GGUF file’s own hyperparameter metadata, following llama.cpp’s general.architecture-prefixed key convention ({arch}.block_count, {arch}.embedding_length, {arch}.attention.head_count, {arch}.expert_count, …) rather than requiring a hand-written preset to already match the file’s shape exactly. This is what lets frink-server (and frink run-real) load an arbitrary checkpoint, not just the three hand-tuned presets in config.rs.

Fields with no corresponding metadata key fall back to widely-used llama.cpp defaults (documented inline) and are listed in the returned config’s best_effort_fields, following the same confirmed-vs-estimated discipline as the hand-written presets.

Trait Implementations

impl Clone for ModelConfig

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.