Trait BackendCostModel 

pub trait BackendCostModel: Send + Sync {
    // Required methods
    fn device(&self) -> Device;
    fn sgemm_gflops(&self, m: usize, k: usize, n: usize) -> f64;
    fn dispatch_overhead_ns(&self) -> f64;
    fn roundtrip_overhead_ns(&self) -> f64;
    fn memory_bw(&self) -> f64;
    fn num_threads(&self) -> usize;

    // Provided methods
    fn host_readback_bw(&self) -> f64 { ... }
    fn unified_memory(&self) -> bool { ... }
}

Hardware-aware cost characteristics for a backend on the current machine.

Required Methods

fn device(&self) -> Device

Identify which device this model is for.

fn sgemm_gflops(&self, m: usize, k: usize, n: usize) -> f64

Effective f32 sgemm throughput in GFLOP/s for the most-used kernel path at the given dimensions. Backends should return their best sustained rate (not peak).

fn dispatch_overhead_ns(&self) -> f64

Cost to dispatch one kernel (function call, BLAS setup, etc.) in ns.

fn roundtrip_overhead_ns(&self) -> f64

Cost to commit + wait for a command buffer / forward pass in ns. Roughly amortized per-forward overhead independent of kernel count.

fn memory_bw(&self) -> f64

Memory bandwidth in bytes/ns (== GB/s).

fn num_threads(&self) -> usize

Number of compute threads available.

Provided Methods

fn host_readback_bw(&self) -> f64

Effective host readback bandwidth (bytes/ns). Defaults to Self::memory_bw.

fn unified_memory(&self) -> bool

True when device and host share the same physical memory (Apple Silicon Metal).

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl BackendCostModel for CpuCostModel

Available on crate feature cpu only.

impl BackendCostModel for WgpuCostModel

Available on crate feature gpu only.