Struct QuantLinear 

pub struct QuantLinear<B: Backend + BackendQuantGguf> { /* private fields */ }

Linear projection backed by a GGUF k-quant weight.

forward() is a tail-call to the inner backend-specific Linear (Metal: MetalGgufLinear, CPU: CpuGgufLinear). LTO inlines through the dispatch.

Implementations

impl<B: Backend + BackendQuantGguf> QuantLinear<B>

pub fn from_gguf_bytes( kind: GgufQuantType, bytes: &[u8], out_features: usize, in_features: usize, ) -> Result<Self>

Build from raw GGUF block bytes.

kind: which k-quant flavour the bytes encode (Q4_K, Q5_K, …). bytes: the on-disk payload, sized by the kind’s block layout.

pub fn from_gguf_fused( parts: &[(GgufQuantType, &[u8], usize)], in_features: usize, ) -> Result<Self>

Build a fused projection from multiple (kind, bytes, rows) parts that share in_features. Each part stays in its own QuantStore (no byte-concat); forward dispatches one matvec per part. Used for Qwen3 qkv_proj when q+k are Q4_K and v is Q6_K — the homogeneous fused-Q4 fast path would have to fall back to eager-fp32, blowing 100 MB per layer.

Trait Implementations

impl<B: Backend + BackendQuantGguf> Linear<B> for QuantLinear<B>

fn in_features(&self) -> usize

fn out_features(&self) -> usize

fn forward( &self, ctx: &mut B::Context, input: &B::Buffer, out: &mut B::Buffer, m: usize, )

Append GEMM work onto ctx. Caller flushes the context when results must be materialised.