Enum BackendPlan 

pub enum BackendPlan {
    Cpu,
    Metal,
    MetalCpuSplit {
        gpu_layers: usize,
        total_layers: usize,
    },
    Cuda,
}

The backend(s) to use for a given model.

Variants

Cpu

Run everything on the CPU using NEON/AVX2 kernels.

Metal

Run everything on the Metal GPU via MLX (Apple Silicon only).

MetalCpuSplit

Run the first gpu_layers transformer layers on Metal, the rest on CPU. Only used when the model doesn’t fit entirely in the Metal memory budget.

Fields

gpu_layers: usize

total_layers: usize

Cuda

CUDA backend (future — not yet implemented).

Implementations

impl BackendPlan

pub fn label(&self) -> String

Human-readable label for display.

pub fn estimated_tps(&self, model_bytes: u64, profile: &DeviceProfile) -> f64

Rough tok/s estimate for a model with model_bytes weights.

Trait Implementations

impl Clone for BackendPlan

fn clone(&self) -> BackendPlan

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BackendPlan

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.