Enum LlmBackendDispatch 

pub enum LlmBackendDispatch {
    Cpu(CpuLlmBackend),
    Metal(MetalLlmBackend),
}

Variants

Cpu(CpuLlmBackend)

Metal(MetalLlmBackend)

Implementations

impl LlmBackendDispatch

pub fn from_kind(kind: LlmBackendKind) -> Result<Self>

Trait Implementations

impl Clone for LlmBackendDispatch

fn clone(&self) -> LlmBackendDispatch

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlmBackendDispatch

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LlmBackend for LlmBackendDispatch

fn name(&self) -> &'static str

fn linear_3d(&self, x: &Tensor, weight: &Tensor) -> Result<Tensor>

fn rms_norm(&self, x: &Tensor, weight: &Tensor, eps: f32) -> Result<Tensor>

fn layer_norm( &self, x: &Tensor, weight: &Tensor, bias: Option<&Tensor>, eps: f32, ) -> Result<Tensor>

fn silu(&self, x: &Tensor) -> Result<Tensor>

fn gelu(&self, x: &Tensor) -> Result<Tensor>

fn add(&self, a: &Tensor, b: &Tensor) -> Result<Tensor>

fn mul(&self, a: &Tensor, b: &Tensor) -> Result<Tensor>

fn apply_rope_positions( &self, x: &Tensor, positions: &[usize], base: f32, ) -> Result<Tensor>

fn gqa_attention( &self, q: &Tensor, k: &Tensor, v: &Tensor, n_kv_heads: usize, causal: bool, ) -> Result<Tensor>

fn logits_from_hidden( &self, hidden: &Tensor, lm_head: &Tensor, ) -> Result<Vec<f32>>

fn linear_3d_bias( &self, x: &Tensor, weight: &Tensor, bias: Option<&Tensor>, ) -> Result<Tensor>

Linear projection with an optional bias added over the last dimension. Backend-agnostic: computes linear_3d then folds in the bias on the host, so every backend gets correct bias handling for free.

fn apply_rope_partial( &self, x: &Tensor, positions: &[usize], base: f32, rotary_dim: usize, ) -> Result<Tensor>

RoPE over only the first rotary_dim channels (Phi partial rotary). Computed on the CPU reference kernel for all backends — it is cheap and avoids backend-specific partial-rotary support.