Struct InferenceOverrides 

pub struct InferenceOverrides {

    pub tree_budget: Option<usize>,
    pub draft_lookahead: Option<usize>,
    pub parallel_threshold: Option<usize>,
    pub early_exit_patience: Option<usize>,
    pub mtp_activation_threshold: Option<usize>,
    pub mtp_cluster_vocab_threshold: Option<usize>,
    pub mtp_shared_kv_prompt_threshold: Option<usize>,
    pub mtp_cluster_size: Option<usize>,
    pub mtp_min_output_tokens: Option<usize>,
    pub mtp_cluster_topk: Option<usize>,
    pub width_rollouts: Option<usize>,
    pub mls_layers: Option<usize>,
    pub max_plan_horizon: Option<usize>,
    pub drafter_lora_path: Option<PathBuf>,
    pub screening_threshold: Option<f32>,
    pub temperature: Option<f32>,
    pub sparse_threshold: Option<f32>,
    pub early_exit_gap: Option<f32>,
    pub sp_kv_threshold: Option<f32>,
    pub early_stop_threshold: Option<f32>,
    pub convergence_selector: Option<ConvergenceSelector>,
    pub depth_tier: Option<DepthTier>,
}

Override DTO for applying per-domain inference budget to a Config.

All fields are Option — None means “keep Config’s current value”. This is a plain struct (no serde) to keep katgpt-core dependency-free from router/TOML types. Conversion from the router’s InferenceBudget happens at the router boundary.

Note: decode_strategy is NOT included here because it depends on project-specific types. Each project handles it at the call site.

See Plan 026 (AutoTTS Dynamic Inference Budget).

Fields

tree_budget: Option<usize>

draft_lookahead: Option<usize>

parallel_threshold: Option<usize>

early_exit_patience: Option<usize>

mtp_activation_threshold: Option<usize>

mtp_cluster_vocab_threshold: Option<usize>

mtp_shared_kv_prompt_threshold: Option<usize>

mtp_cluster_size: Option<usize>

mtp_min_output_tokens: Option<usize>

Minimum expected output tokens for MTP (Plan 117 T15). When overridden, skips MTP when remaining tokens < threshold.

mtp_cluster_topk: Option<usize>

Top-K cluster selection for clustered LM head (Plan 117 T22). When K > 1, compute logits for tokens in top-K clusters instead of just top-1.

width_rollouts: Option<usize>

mls_layers: Option<usize>

max_plan_horizon: Option<usize>

drafter_lora_path: Option<PathBuf>

screening_threshold: Option<f32>

temperature: Option<f32>

sparse_threshold: Option<f32>

early_exit_gap: Option<f32>

sp_kv_threshold: Option<f32>

early_stop_threshold: Option<f32>

convergence_selector: Option<ConvergenceSelector>

depth_tier: Option<DepthTier>

Override depth tier for layer count capping at inference time. When set, caps the layer loop to tier.max_layers(). None = use all layers (backward compatible).

Trait Implementations

impl Clone for InferenceOverrides

fn clone(&self) -> InferenceOverrides

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InferenceOverrides

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for InferenceOverrides

fn default() -> InferenceOverrides

Returns the “default value” for a type.