Skip to main content

sqnbit_gemm

Function sqnbit_gemm 

Source
pub fn sqnbit_gemm(
    packed: &SQNBitPackedB,
    m: usize,
    a: &[f32],
    bias: Option<&[f32]>,
    c: &mut [f32],
    multithread: bool,
)
Expand description

Compute C = A * dequant(packed) + bias for row-major A (m x k) and C (m x n), reusing a pre-packed blockwise-quantized weight.

When multithread is true MLAS partitions the GEMM across the current Rayon pool (see PackedB threading notes); otherwise it runs serially. bias, when present, is added by MLAS itself (length n).