Struct QuantLinear 

pub struct QuantLinear<B: Backend> { /* private fields */ }

Linear projection backed by a GGUF k-quant weight kept quantised in backend memory.

forward calls into Backend::gemm_quant, which dequants the weight into a transient fp16 buffer (Metal) or pre-dequanted fp32 weights (CPU) and then runs the matmul. See B::QuantStore per backend for the storage format details.

Future k-quant flavours (Q5_K, Q6_K, Q8_0) plug in via the GgufQuantType discriminator passed to the constructor — no new QuantLinear type required.

Implementations

impl<B: Backend> QuantLinear<B>

pub fn from_gguf_bytes( kind: GgufQuantType, bytes: &[u8], out_features: usize, in_features: usize, ) -> Result<Self>

Build from raw GGUF block bytes.

kind: which k-quant flavour the bytes encode (Q4_K, Q5_K, …). bytes: the on-disk payload, sized by the kind’s block layout.

pub fn from_gguf_fused( parts: &[(GgufQuantType, &[u8], usize)], in_features: usize, ) -> Result<Self>

Build a fused projection from multiple (kind, bytes, rows) parts that share in_features. Each part stays in its own QuantStore (no byte-concat); forward dispatches one matvec per part. Used for Qwen3 qkv_proj when q+k are Q4_K and v is Q6_K — the homogeneous fused-Q4 fast path would have to fall back to eager-fp32, blowing 100 MB per layer.

pub fn from_store( store: B::QuantStore, out_features: usize, in_features: usize, ) -> Self

For tests / advanced callers that have already constructed a B::QuantStore (e.g. through the Backend’s own ingestion path).

pub fn store(&self) -> &B::QuantStore

Trait Implementations

impl<B: Backend> Linear<B> for QuantLinear<B>

fn in_features(&self) -> usize

fn out_features(&self) -> usize

fn forward( &self, ctx: &mut B::Context, input: &B::Buffer, out: &mut B::Buffer, m: usize, )

Append GEMM work onto ctx. Caller flushes the context when results must be materialised.