Struct SQNBitPackedB 

pub struct SQNBitPackedB { /* private fields */ }

MLAS-packed blockwise-quantized B weight for sqnbit_gemm, mirroring how ORT pre-packs the constant MatMulNBits initializer once and reuses it.

The B bytes, scales, and optional zero points use the standard ONNX MatMulNBits layout ([N, ceil(K/blk_len), blk_len*bits/8], LSB-first nibbles; scales [N, ceil(K/blk_len)]; packed uint8 zero points). For Fp32 compute MLAS repacks only the nibbles and consumes scales/zero points at GEMM time (kept here so the packed weight is self-contained). For Int8 compute MLAS bakes scale and zero point into per-block sums inside the packed buffer, so scale/zp are unused at GEMM time. A default (absent) zero point is the ONNX/MLAS midpoint (8 for int4), so symmetric weights need no zero point.

Implementations

impl SQNBitPackedB

pub fn new( n: usize, k: usize, bits: usize, blk_len: usize, comp: SQNBitComputeType, quant_b_data: &[u8], scale: &[f32], zp: Option<&[u8]>, ) -> Option<Self>

Pack a blockwise-quantized B weight, returning None when MLAS reports no packing/kernel is available for this shape on the current host (the caller must then fall back to another path).

pub fn dimensions(&self) -> (usize, usize)

Logical (k, n) dimensions of the packed weight.

Trait Implementations

impl Drop for SQNBitPackedB

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl Send for SQNBitPackedB

impl Sync for SQNBitPackedB