Skip to main content

quantize_bit4

Function quantize_bit4 

Source
pub fn quantize_bit4(
    v: &[f32],
    out: &mut [u8],
) -> Result<Bit4Factors, QuantError>
Expand description

Quantizes one row to four-bit Extended-RaBitQ codes.

Two coordinates occupy each byte in coordinate order, high nibble first. Unsigned code u represents u - 7.5. For an odd dimension, the unused low nibble in the final byte is canonical zero padding.

ยงErrors

Returns QuantError::EmptyVector for an empty input, QuantError::DimensionTooLarge above the i8 scoring-kernel limit, QuantError::NonFinite for NaN or infinity, and QuantError::OutputLength unless out.len() == ceil(v.len() / 2). Validation completes before out is modified.