Module quant 

INT4 weight quantization/dequantization, bit-for-bit parity with Python quantize.py.

Scheme: symmetric group-wise INT4, group_size=32 along input axis. Scale = max(|group|) / 7.0, clamped >= 1e-8. Values packed as two nibbles per byte: low nibble = even row, high nibble = odd row. Stored signed as two’s-complement 4-bit: range [-8, 7].

Data layout: row-major (pair-major) — data[pair * out_feat + o]. This makes the inner loop over output features contiguous, enabling AVX2 auto-vectorization in matvec_inner_avx2 and plain scalar vectorization elsewhere.

Structs

QuantizedWeight

Packed INT4 weight tensor with per-group f32 scales.

Constants

GROUP_SIZE

SCALE_MIN

Functions

has_avx2