Struct Config 

pub struct Config {

    pub vocab_size: usize,
    pub block_size: usize,
    pub n_embd: usize,
    pub n_head: usize,
    pub head_dim: usize,
    pub mlp_hidden: usize,
    pub n_layer: usize,
    pub n_kv_head: usize,
    pub bos_token: usize,
    pub draft_lookahead: usize,
    pub tree_budget: usize,
    pub parallel_threshold: usize,
    pub lora_rank: usize,
    pub early_exit_patience: usize,
    pub mtp_activation_threshold: usize,
    pub mtp_cluster_vocab_threshold: usize,
    pub mtp_shared_kv_prompt_threshold: usize,
    pub mtp_cluster_size: usize,
    pub mtp_min_output_tokens: usize,
    pub mtp_cluster_topk: usize,
    pub mask_token: usize,
    pub sp_kv_window: usize,
    pub sp_kv_predictor_hidden: usize,
    pub width_rollouts: usize,
    pub d2f_block_size: usize,
    pub mls_layers: usize,
    pub rms_norm_eps: f64,
    pub sp_kv_predictor_lr_mult: f32,
    pub temperature: f32,
    pub lora_alpha: f32,
    pub lora_dropout: f32,
    pub screening_threshold: f32,
    pub sparse_threshold: f32,
    pub early_exit_gap: f32,
    pub hla_decay: f32,
    pub rope_theta: f32,
    pub attn_logit_softcapping: f32,
    pub final_logit_softcapping: f32,
    pub sp_kv_threshold: f32,
    pub early_stop_threshold: f32,
    pub parallax_gate_scale: f32,
    pub emotion_desperation_threshold: f32,
    pub lora_targets: Vec<String>,
    pub hla_mode: HlaMode,
    pub model_arch: ModelArchitecture,
    pub attention_mode: AttentionMode,
    pub convergence_selector: ConvergenceSelector,
    pub loop_mode: LoopMode,
    pub hybrid_pattern: HybridPattern,
    pub loop_min: usize,
    pub loop_max: usize,
    pub weight_dtype: WeightDtype,
    pub hla_normalize: bool,
    pub rms_norm_offset: bool,
    pub tied_embeddings: bool,
    pub use_rope: bool,
    pub post_norm: bool,
    pub gated_attn: bool,
    pub parallax_zero_init: bool,
}

Transformer model configuration — superset of both katgpt-rs and riir-engine.

Fields are ordered by descending alignment to minimize padding: usize/u64 → f64 → enums (usize-discriminant) → f32 → Vec → u16 → u8/bool.

Fields

vocab_size: usize

block_size: usize

n_embd: usize

n_head: usize

head_dim: usize

mlp_hidden: usize

n_layer: usize

n_kv_head: usize

bos_token: usize

draft_lookahead: usize

tree_budget: usize

parallel_threshold: usize

lora_rank: usize

early_exit_patience: usize

mtp_activation_threshold: usize

mtp_cluster_vocab_threshold: usize

mtp_shared_kv_prompt_threshold: usize

mtp_cluster_size: usize

mtp_min_output_tokens: usize

Minimum expected output tokens for MTP speculative decoding. If remaining tokens < this threshold, MTP is skipped (single-token path). Prevents MoE overhead on short texts (Plan 117 Phase 2).

mtp_cluster_topk: usize

Top-K cluster selection for clustered LM head (Plan 117 T20). When K > 1, compute logits for tokens in top-K clusters instead of just top-1. Default 1 = backward compatible (single cluster = current behavior).

mask_token: usize

sp_kv_window: usize

sp_kv_predictor_hidden: usize

width_rollouts: usize

d2f_block_size: usize

mls_layers: usize

Number of last layers to sum before LM head. 0 = disabled (standard). (Plan 104: Research 68)

rms_norm_eps: f64

sp_kv_predictor_lr_mult: f32

temperature: f32

lora_alpha: f32

lora_dropout: f32

screening_threshold: f32

sparse_threshold: f32

early_exit_gap: f32

hla_decay: f32

rope_theta: f32

attn_logit_softcapping: f32

final_logit_softcapping: f32

sp_kv_threshold: f32

early_stop_threshold: f32

parallax_gate_scale: f32

Parallax covariance correction gate scale. 0.0 = disabled (pure softmax), 1.0 = full correction. Only meaningful when parallax_attn feature is enabled and R projection weights are loaded.

emotion_desperation_threshold: f32

Desperation score threshold for emotion-aware session flagging (Plan 162 T12). When the mean desperation projection exceeds this value, is_desperate_session() returns true. Default: 0.5 (moderate desperation). Range: [0.0, 1.0].

lora_targets: Vec<String>

hla_mode: HlaMode

model_arch: ModelArchitecture

attention_mode: AttentionMode

convergence_selector: ConvergenceSelector

loop_mode: LoopMode

hybrid_pattern: HybridPattern

loop_min: usize

loop_max: usize

weight_dtype: WeightDtype

hla_normalize: bool

rms_norm_offset: bool

tied_embeddings: bool

use_rope: bool

post_norm: bool

gated_attn: bool

parallax_zero_init: bool

Whether W_R starts zeroed (true = recover exact softmax at init).

Implementations

impl Config

pub fn effective_loop_count(&self, elastic_override: Option<usize>) -> usize

Compute the effective loop count for forward_looped, applying an optional elastic override clamped to [loop_min, 2×loop_max].

(Issue 035, Research 273 — ELT arXiv:2604.09168 Any-Time inference.)

• elastic_override = None → use loop_mode’s natural loop count (byte-identical to pre-Issue-035 behavior).
• elastic_override = Some(L) with LoopMode::WeightShared → clamp L to [max(loop_min, 1), 2 × max(loop_max, base)].
  • Below loop_min (default 1): clamped up. ELT §1.4 establishes a minimum depth for representational capacity (1N × 32L collapsed to FID 10.30 vs 2.83 for 16N × 2L).
  • Above 2 × loop_max: clamped down. ELT §1.5 shows modest over-looping beyond L_max is regularized (UCF-101 peak FVD at L=6 with L_max=4), but quality eventually deteriorates — 2× is the cap.
• elastic_override = Some(_) with LoopMode::None or TrainingFree → refused (returns base); there is no weight-shared loop to elastically exit from.

loop_max == 0 is a sentinel meaning “derive from loop_mode” (use WeightShared.loop_count). This keeps the 12 existing Config constructors unchanged in semantics — they default to loop_max: 0 which resolves to the natural loop count.

pub fn micro() -> Self

Micro GPT config matching talos-vs-macbook reference: vocab=27, block=16, n_layer=1, n_head=4, n_embd=16, head_dim=4, RMSNorm (no learnable gain), ReLU MLP (4x), no biases, untied lm_head.

pub fn micro_lora() -> Self

Micro config with LoRA defaults (Plan 008).

pub fn micro_dllm() -> Self

Micro config for Discrete Diffusion Language Model training (Plan 068: D2F). Bidirectional attention by default, mask_token = vocab_size - 1.

pub fn game() -> Self

Game config for Bomberman LoRA training (Plan 041). Tiny Transformer for board state → action prediction. 10-token vocab: 4 board cells (0-3) + 6 actions (4-9). 170-token sequences: 169 board cells + 1 action. ~18K params total, ~1.5K LoRA params (rank=4).

pub fn game_go() -> Self

Game config for Go 9×9 LoRA training (Plan 078). Tiny Transformer for board state → move prediction. 85-token vocab: 3 board cells (Empty=0, Black=1, White=2) + 81 positions (3..83) + 1 pass (84). 82-token sequences: 81 board cells + 1 action. ~16K params total, ~1.3K LoRA params (rank=4).

pub fn game_fft() -> Self

Game config for FFT Tactics Arena LoRA training (Plan 296 T7.3). Tiny Transformer for battle state → action prediction.

Token Layout

• State vocab (values 0..9): team(0-1), class(0-5), hp_bucket(0-7), mp_bucket(0-3), pos_x(0-7), pos_y(0-7), alive(0-1).
• Action tokens: 10..19 (9 FFT ActionTypes).

Sequence (58 tokens): [tick, u0_team, u0_class, u0_hp, u0_mp, u0_x, u0_y, u0_alive, u1_..., ..., u7_..., action_token]

Per-unit = 7 tokens × 8 units = 56, +1 tick +1 action = 58 tokens. ~18K params total, ~1.5K LoRA params (rank=4). Comparable to Bomber/Go.

pub fn draft() -> Self

Lightweight draft model for speculative decoding (~4× smaller than target). Same vocab/block to share embeddings, but embd=4, heads=2, mlp=16.

pub fn small_target() -> Self

Small target model for multi-layer testing. vocab=4096, block=256, n_layer=4, n_head=4, n_embd=64, head_dim=16, MLP hidden=256.

pub fn gqa_draft() -> Self

GQA draft config: 8 Q heads, 2 KV heads (4:1 ratio, 4× KV cache reduction).

pub fn bpe() -> Self

BPE tokenizer config for Rust source code. vocab=4096, block=256, n_layer=1, n_head=4, n_embd=32, head_dim=8, MLP hidden=128.

pub fn bpe_draft() -> Self

BPE draft model (smaller for speculative decoding). Same vocab/block as bpe(), but embd=16, heads=2, mlp=64.

pub fn gemma2_2b() -> Self

Gemma 2 2B config for real model inference (Plan 087). hidden_size=2304, intermediate_size=9216, vocab=256000, layers=26, heads=8, kv_heads=4, head_dim=256, max_seq=8192. Uses GeGLU MLP, RoPE, RMSNorm offset, tied embeddings, post-norm.

pub fn validate(&self) -> Result<(), String>

Validate config consistency. Returns Err with message on invalid config.

pub fn with_overrides(self, overrides: &InferenceOverrides) -> Self

None fields are left unchanged; Some fields replace the current value. Used by the router to inject domain-specific budgets from TOML config.

Trait Implementations

impl Clone for Config

fn clone(&self) -> Config

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for Config

fn default() -> Self

Returns the “default value” for a type.