Struct GgufLinear 

pub struct GgufLinear<B: Backend> { /* private fields */ }

Linear projection backed by a GGUF-sourced quantized tensor.

Internally a DenseLinear<B> (Phase 1B), so the runtime path is the same as a plain dense weight. The distinct type lets later phases evolve the representation without changing call sites.

Implementations

impl<B: Backend> GgufLinear<B>

pub fn from_qtensor(qt: &QTensor) -> CandleResult<Self>

Build from a candle QTensor previously read out of a GGUF file.

Expects a 2-D weight whose shape is [out_features, in_features] (the GGUF convention for linear projections — rows are output neurons). Errors if the rank is wrong or the dequant step fails.

pub fn from_qtensor_with_bias( qt: &QTensor, bias_qt: &QTensor, ) -> CandleResult<Self>

Build with a bias vector. bias_qt must be a 1-D [out_features] tensor — typical for Qwen2.5 / Bert / any model with attention bias.

pub fn from_dense_rows( weight_row_major: &[f32], out_features: usize, in_features: usize, ) -> Self

Build directly from already-dequantized fp32 weights. Useful when the caller has already paid the dequant cost (e.g. cached weights, or constructing from synthetic data in tests).

Trait Implementations

impl<B: Backend> Linear<B> for GgufLinear<B>

fn in_features(&self) -> usize

fn out_features(&self) -> usize

fn forward( &self, ctx: &mut B::Context, input: &B::Buffer, out: &mut B::Buffer, m: usize, )

Append GEMM work onto ctx. Caller flushes the context when results must be materialised.