Struct QuantizedBytes 

pub struct QuantizedBytes {
    pub bytes: Bytes,
    pub scheme: QuantScheme,
    pub shape: Shape,
}

Quantized data bytes representation.

Notes

1. The quantized values are laid out as the scheme’s store says: a byte each for Native, or packed into little-endian words along the packed axis as cubecl stores them, each line padded to whole words.
2. Quantization parameters are appended to the tensor data. As such, the last bytes always correspond to the scale parameter. If the quantization scheme includes an offset (zero-point) parameter, it is next to last.

Fields

bytes: Bytes

The quantized values and quantization parameters represented as bytes.

scheme: QuantScheme

The quantization scheme.

shape: Shape

The shape of the quantized tensor. The block count, and so the scale count, follows from it per axis: a block that does not span the trailing dimension is not a run of elements.

Implementations

impl QuantizedBytes

pub fn new<E: CheckedBitPattern + NoUninit>( value: Vec<E>, shape: impl Into<Shape>, scheme: QuantScheme, scales: &[f32], global: Option<f32>, ) -> Self

Creates a new quantized bytes representation.

value holds one code per element and scales the block scales, both row-major; global is the per-tensor scale, required by a two-level scheme and rejected by a one-level one.

pub fn num_elements(&self) -> usize

The number of quantized elements.

pub fn into_vec_i8(self) -> (Vec<i8>, DecodedScales)

The codes, one i8 per element in row-major order, and the scales; a float code is its bits.