Struct AutoTuner 

pub struct AutoTuner { /* private fields */ }

The auto-tuner: detects hardware and recommends inference parameters.

Implementations

impl AutoTuner

pub fn new() -> Self

Create a new auto-tuner that auto-detects CPU and uses system RSS as a rough memory estimate (defaults to 4096 MB if detection fails).

pub fn with_memory_mb(memory_mb: usize) -> Self

Create an auto-tuner with an explicit memory budget (in MB).

pub fn recommend( &self, model_params: usize, bits_per_weight: f32, num_layers: usize, num_heads: usize, head_dim: usize, ) -> TuningRecommendation

Generate tuning recommendations for a specific model configuration.

• model_params — total number of model parameters.
• bits_per_weight — quantisation bits (e.g. 1.125 for Q1_0_g128).
• num_layers / num_heads / head_dim — transformer architecture.

pub fn benchmark_kernel(&self, iterations: usize) -> KernelBenchmark

Quick micro-benchmark: run a synthetic FMA-heavy kernel to estimate raw compute throughput.

pub fn cpu_features(&self) -> &CpuFeatures

Reference to the detected CPU features.

pub fn report(&self) -> String

Full diagnostic report.

Trait Implementations

impl Default for AutoTuner

fn default() -> Self

Returns the “default value” for a type.