Trait TimingModel 

pub trait TimingModel: Send + Sync {
    // Required methods
    fn predict_prefill_ms(
        &self,
        batch_size: usize,
        mean_isl: usize,
        mean_prefix: usize,
    ) -> Result<f64>;
    fn predict_decode_ms(
        &self,
        batch_size: usize,
        active_kv_tokens: usize,
        mean_context_length: usize,
        total_kv_tokens: usize,
    ) -> Result<f64>;
}

Runtime latency model injected at the engine boundary.

Implementations may call AIC, interpolate profiler data, or use another provider without adding that dependency to aisimulate-core.

Required Methods

fn predict_prefill_ms( &self, batch_size: usize, mean_isl: usize, mean_prefix: usize, ) -> Result<f64>

Predict one prefill batch’s latency in milliseconds.

fn predict_decode_ms( &self, batch_size: usize, active_kv_tokens: usize, mean_context_length: usize, total_kv_tokens: usize, ) -> Result<f64>

Predict one decode batch’s latency in milliseconds.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors