Struct ForwardPassPerfOptions 

pub struct ForwardPassPerfOptions {
    pub max_observations: usize,
    pub min_observations: usize,
    pub min_faster_correction_factor: Option<f64>,
    pub max_slower_correction_factor: Option<f64>,
    pub bucket_count: usize,
    pub max_num_tokens: u32,
    pub max_batch_size: u32,
    pub max_kv_tokens: u32,
}

In-memory tuning controls for ForwardPassPerfModel.

The defaults retain a bounded sliding sample set, wait for enough observations before predicting from learned data, bucket observations by workload kind, and bound native correction factors to [0.5, 2.0].

Fields

max_observations: usize

Maximum retained observations across all buckets for each inferred workload kind.

min_observations: usize

Minimum retained observations required before a regression fit or native correction is used for an inferred workload kind.

min_faster_correction_factor: Option<f64>

Optional absolute lower bound on native correction factors for observations faster than the native estimate.

Values must be finite, greater than 0.0, and at most 1.0. Defaults to 0.5, limiting learned speedups to 2x. Setting this to 1.0 disables faster corrections; setting it to None removes the lower bound. Regression fallback does not use this option.

max_slower_correction_factor: Option<f64>

Optional absolute upper bound on native correction factors for observations slower than the native estimate.

Values must be finite and at least 1.0. Defaults to 2.0, limiting learned slowdowns to 2x. Setting this to 1.0 disables slower corrections; setting it to None removes the upper bound. Regression fallback does not use this option.

bucket_count: usize

Target bucket count for workload-specific sample retirement and correction lookup.

max_num_tokens: u32

Upper bound for the sum_prefill_tokens correction axis.

Used by prefill and mixed/agg workload kinds. The lower bound is always 0.

max_batch_size: u32

Upper bound for the num_decode_requests correction axis.

Used by the decode workload kind. The lower bound is always 0.

max_kv_tokens: u32

Upper bound for the sum_decode_kv_tokens correction axis.

Used by decode and mixed/agg workload kinds. The lower bound is always 0.

Trait Implementations

impl Clone for ForwardPassPerfOptions

fn clone(&self) -> ForwardPassPerfOptions

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ForwardPassPerfOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ForwardPassPerfOptions

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for ForwardPassPerfOptions

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for ForwardPassPerfOptions

fn eq(&self, other: &ForwardPassPerfOptions) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for ForwardPassPerfOptions

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ForwardPassPerfOptions