Enum QuantizationType 

pub enum QuantizationType {
    Int8,
    Int4,
    Int2,
    FP8E4M3,
    FP8E5M2,
    FP16,
    BF16,
    Binary,
    Ternary,
}

Quantization data types.

Variants

Int8

8-bit integer quantization

Int4

4-bit integer quantization

Int2

2-bit integer quantization (extreme compression)

FP8E4M3

8-bit floating point (E4M3 or E5M2)

FP8E5M2

FP8 E5M2 format

FP16

16-bit floating point

BF16

16-bit brain float

Binary

Binary quantization (1-bit)

Ternary

Ternary quantization (-1, 0, 1)

Implementations

impl QuantizationType

pub fn bits(&self) -> u32

Returns the number of bits used by this quantization type.

pub fn compression_ratio(&self) -> f64

Returns the theoretical compression ratio vs FP32.

pub fn is_floating_point(&self) -> bool

Returns whether this type supports floating point values.

Trait Implementations

impl Clone for QuantizationType

fn clone(&self) -> QuantizationType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizationType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for QuantizationType

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Hash for QuantizationType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for QuantizationType

fn eq(&self, other: &QuantizationType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for QuantizationType

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for QuantizationType

impl Eq for QuantizationType

impl StructuralPartialEq for QuantizationType