Struct RuntimeKnobs 

pub struct RuntimeKnobs {

    pub kv_capacity: Option<usize>,
    pub max_model_len: Option<usize>,
    pub chunked_prefill_size: Option<usize>,
    pub batch_decode_prof: bool,
    pub next_batch_prof: bool,
    pub rbd_prof: bool,
    pub unified_post_prof: bool,
    pub prefix_cache_enabled: bool,
    pub model_path: Option<String>,
    pub spec_draft: Option<String>,
    pub spec_n: Option<usize>,
    pub dtype: Option<String>,
    pub metal_dtype: Option<String>,
    pub tp: Option<usize>,
}

Engine runtime knobs the CLI/autosizer resolves and injects via the runtime-config snapshot. The continuous engine reads these from the typed config instead of std::env::vars(), so the env bridge stays at the composition root and tests can vary the knobs per EngineConfig.

Fields

kv_capacity: Option<usize>

max_model_len: Option<usize>

chunked_prefill_size: Option<usize>

batch_decode_prof: bool

next_batch_prof: bool

rbd_prof: bool

unified_post_prof: bool

prefix_cache_enabled: bool

model_path: Option<String>

spec_draft: Option<String>

spec_n: Option<usize>

dtype: Option<String>

metal_dtype: Option<String>

tp: Option<usize>

Trait Implementations

impl Clone for RuntimeKnobs

fn clone(&self) -> RuntimeKnobs

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RuntimeKnobs

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RuntimeKnobs

fn default() -> RuntimeKnobs

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for RuntimeKnobs

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for RuntimeKnobs

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.