Module quantization 

Quantization data representation. Quantization data representation.

Structs

BlockLayout

Which block each element of a tensor falls in, for a block scheme. A block is a rectangle, so its members are a run of the flat storage only when it spans the trailing dimension; anything walking values against block scales asks here rather than chunking.

BlockScale

The per-block scale level of a QuantScheme: one scale per block of values.

BlockSize

Copyable block size, specialized version of SmallVec.

DecodedScales

Scales recovered from a quantized byte buffer.

QParamTensor

A quantization parameter tensor descriptor.

QParams

The quantization tensor data parameters.

QuantConfig

Configuration for a device quantization behavior.

QuantScheme

Describes a quantization scheme/configuration.

QuantizedBytes

Quantized data bytes representation.

Enums

Calibration

Calibration method used to compute the quantization range mapping.

QuantAcc

The precision of accumulating elements.

QuantMode

Strategy used to quantize values.

QuantPropagation

Specify if the output of an operation is quantized using the scheme of the input or returned unquantized.

QuantStore

Data type used to stored quantized values.

QuantValue

Data type used to represent quantized values.

ScaleDtype

The data type a scale level stores its scales in.

Constants

QPARAM_ALIGN

Alignment (in bytes) for quantization parameters in serialized tensor data.

Functions

global_scale_dtype

The dtype of the per-tensor scale block scales are normalized against, for a two-level scheme.

global_scale_size

Bytes taken by the per-tensor scale, zero for a scheme that does not carry one over blocks.

pack_i8s_to_u32s

Pack signed 8-bit integer values into a sequence of unsigned 32-bit integers.

params_shape

Calculate the shape of the block scale grid for a given tensor and scheme.

quantizable

Whether a backend can quantize against this scheme’s scales.

quantized_data_len

Total bytes a tensor of shape occupies under scheme, laid out as QuantizedBytes::new writes it: values, then block scales, then (for a two-level scheme) the per-tensor scale.

scale_size

Bytes per stored scale entry for the given scale dtype.

scale_to_dtype

Round a scale up to the smallest value representable by the scale dtype that is no smaller.