Struct Costs 

pub struct Costs {
    pub dev_bytes_per_s: f64,
    pub cpu_bytes_per_s: f64,
    pub dyn_layer_s: f64,
    pub graph_submit_s: f64,
    pub cpu_layer_s: f64,
    pub fill_bytes_per_s: f64,
    pub hit_curve: Vec<(f64, f64)>,
}

Measured cost constants of the decision.

Fields

dev_bytes_per_s: f64

Effective device bytes/s for a resident weight stream.

cpu_bytes_per_s: f64

Effective host bytes/s for q4tp expert / q8_2f projection streams.

dyn_layer_s: f64

Fixed cost of one host-walked layer whose projections and resident experts run per-op on the device: the QKV and O submits, the MoE frame fence, the host attention core and route.

graph_submit_s: f64

Fixed cost of one whole-token graph submit.

cpu_layer_s: f64

Fixed cost of one pure-host layer (pool dispatch barriers).

fill_bytes_per_s: f64

Upload bytes/s of a bank fill.

hit_curve: Vec<(f64, f64)>

Per-layer LRU hit rate of a decode stream by slots per layer (trace replay of real routing). Interpolated in log(slots).

Implementations

impl Costs

pub fn measured() -> Self

RTX PRO 6000 Blackwell (96 GB, Vulkan) + EPYC 9655 in a 24-core cgroup, MiMo-V2.6-Flash q4tp (runs of 2026-09-24):

• dyn_layer_s: a bank-served layer measured 0.9–1.0 ms of host walk per layer at ~100 % hits with the generic frame (QKV submit 0.22 ms, O 0.14 ms, frame 0.43 ms, attention core 0.06 ms); the dedicated bank kernels take ~0.2 ms off the frame. The device’s own bytes are charged separately at dev_bytes_per_s.
• hit_curve: per-layer LRU hit rate after a 64-token warm-up, replayed from a 446-token CMF_MOE_TRACE of docs/ppl_nat.txt (tools/moe_lru_sim.py).
• cpu_bytes_per_s: q4tp/q8_2f streaming of the host walk with the 22-thread pool.

pub fn hit_rate(&self, slots: f64, n_experts: usize) -> f64

LRU hit rate at slots per layer.

Trait Implementations

impl Clone for Costs

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Costs

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.