Trait BackendCostModel 

pub trait BackendCostModel: Send + Sync {
    // Required methods
    fn device(&self) -> Device;
    fn sgemm_gflops(&self, m: usize, k: usize, n: usize) -> f64;
    fn dispatch_overhead_ns(&self) -> f64;
    fn roundtrip_overhead_ns(&self) -> f64;
    fn memory_bw(&self) -> f64;
    fn num_threads(&self) -> usize;
}

Hardware-aware cost characteristics for a backend on the current machine.

Required Methods

fn device(&self) -> Device

Identify which device this model is for.

fn sgemm_gflops(&self, m: usize, k: usize, n: usize) -> f64

Effective f32 sgemm throughput in GFLOP/s for the most-used kernel path at the given dimensions. Backends should return their best sustained rate (not peak).

fn dispatch_overhead_ns(&self) -> f64

Cost to dispatch one kernel (function call, BLAS setup, etc.) in ns.

fn roundtrip_overhead_ns(&self) -> f64

Cost to commit + wait for a command buffer / forward pass in ns. Roughly amortized per-forward overhead independent of kernel count.

fn memory_bw(&self) -> f64

Memory bandwidth in bytes/ns (== GB/s).

fn num_threads(&self) -> usize

Number of compute threads available.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl BackendCostModel for CpuCostModel

Available on crate feature cpu only.