Enum QuantizationMethod 

pub enum QuantizationMethod {
    Uniform,
    Symmetric,
    Affine,
    PowerOfTwo,
    Int4,
    UInt4,
    Float16,
    BFloat16,
    PerChannelSymmetric,
    PerChannelAffine,
}

Supported methods of quantization

Variants

Uniform

Uniform quantization maps the input range to uniform discrete levels with equal spacing between consecutive levels

Symmetric

Symmetric quantization is centered around zero and has equal positive and negative ranges, making it suitable for weight matrices

Affine

Affine quantization uses the formula q = scale * (x - zero_point) allowing better representation of asymmetric distributions

PowerOfTwo

Power-of-two quantization uses powers of 2 for the scale factor, enabling efficient implementation with bitshifts

Int4

Int4 quantization uses 4-bit signed integers, packing two values into each byte for memory efficiency. This is useful for model compression in ML applications.

UInt4

UInt4 quantization uses 4-bit unsigned integers, packing two values into each byte. This provides a positive-only range with maximum memory efficiency.

Float16

Float16 quantization uses IEEE 754 16-bit half-precision floating point format. It provides a good balance between precision and memory efficiency for ML models.

BFloat16

BFloat16 quantization uses the “brain floating point” 16-bit format, which has the same exponent size as f32 but fewer mantissa bits. This is especially well-suited for deep learning applications.

PerChannelSymmetric

Per-channel symmetric quantization applies different symmetric quantization parameters to each channel (column), improving accuracy for matrices with varying distributions across channels.

PerChannelAffine

Per-channel affine quantization applies different affine quantization parameters to each channel (column), allowing for better representation of asymmetric distributions that vary by channel.

Trait Implementations

impl Clone for QuantizationMethod

fn clone(&self) -> QuantizationMethod

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizationMethod

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for QuantizationMethod

fn eq(&self, other: &QuantizationMethod) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for QuantizationMethod

impl Eq for QuantizationMethod

impl StructuralPartialEq for QuantizationMethod