Struct BackendSelector 

pub struct BackendSelector { /* private fields */ }

Cost model for backend selection

Based on Gregg & Hazelwood (2011) 5× PCIe rule for GPU dispatch decisions.

Implementations

impl BackendSelector

pub fn new() -> Self

Create a new backend selector with default thresholds

pub fn with_pcie_bandwidth(self, bandwidth: f64) -> Self

Configure custom PCIe bandwidth

pub fn with_gpu_gflops(self, gflops: f64) -> Self

Configure custom GPU throughput

pub fn with_min_dispatch_ratio(self, ratio: f64) -> Self

Configure custom dispatch ratio threshold

pub fn select_backend(&self, data_bytes: usize, flops: u64) -> Backend

Select optimal backend based on workload characteristics

Arguments

• data_bytes - Amount of data to transfer (host → device)
• flops - Floating point operations required

Returns

Recommended backend based on cost model

Cost Model

GPU dispatch is beneficial when:

compute_time > min_dispatch_ratio × transfer_time

Per Gregg & Hazelwood (2011), the 5× rule accounts for:

• Host→Device transfer (PCIe overhead)
• Kernel launch latency
• Device→Host transfer
• CPU-GPU synchronization

pub fn select_for_matmul(&self, m: usize, n: usize, k: usize) -> Backend

Select backend for matrix multiplication

Arguments

• m, n, k - Matrix dimensions (M×K) × (K×N) = (M×N)

Complexity

• Data: O(mk + kn + mn) bytes
• FLOPs: O(2mnk) operations

pub fn select_for_vector_op(&self, n: usize, ops_per_element: u64) -> Backend

Select backend for vector operations

Arguments

• n - Vector length
• ops_per_element - Operations per element (e.g., 2 for dot product)

pub fn select_for_elementwise(&self, n: usize) -> Backend

Select backend for element-wise operations

Element-wise ops are memory-bound, so GPU is rarely beneficial

pub fn select_with_moe( &self, complexity: OpComplexity, data_size: usize, ) -> Backend

MoE (Mixture-of-Experts) routing: select backend based on operation complexity

Arguments

• complexity - Operation complexity (Low/Medium/High)
• data_size - Number of elements in the operation

Returns

Recommended backend using adaptive thresholds per complexity level

MoE Thresholds (per empirical performance analysis)

• Low complexity (element-wise): SIMD at 1M+ elements, never GPU
• Medium complexity (reductions): SIMD at 10K+, GPU at 100K+ elements
• High complexity (matmul): SIMD at 1K+, GPU at 10K+ elements

pub fn selection_stats( &self, complexity: OpComplexity, data_size: usize, ) -> SelectionStats

Get selection statistics for profiling

Trait Implementations

impl Clone for BackendSelector

fn clone(&self) -> BackendSelector

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BackendSelector

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for BackendSelector

fn default() -> Self

Returns the “default value” for a type.