Struct CpuGptqLinear 

pub struct CpuGptqLinear {
    pub weight_f32: Vec<f32>,
    pub bias: Option<Vec<f32>>,
    pub in_features: usize,
    pub out_features: usize,
}

CPU GPTQ Linear: holds dequantized fp32 weights [out_features, in_features] row-major, optional bias [out_features], dispatches via CpuBackend::gemm.

The dequantization happens once in BackendQuantMarlin::load_gptq — inference is just a regular f32 GEMM.

Fields

weight_f32: Vec<f32>

bias: Option<Vec<f32>>

in_features: usize

out_features: usize

Trait Implementations

impl Linear<CpuBackend> for CpuGptqLinear

fn in_features(&self) -> usize

fn out_features(&self) -> usize

fn forward( &self, ctx: &mut <CpuBackend as Backend>::Context, input: &<CpuBackend as Backend>::Buffer, out: &mut <CpuBackend as Backend>::Buffer, m: usize, )

Append GEMM work onto ctx. Caller flushes the context when results must be materialised.