Enum QuantStrategy 

pub enum QuantStrategy {
    None,
    INT8WeightOnly,
    FP16,
    BF16,
    INT8Dynamic,
    MixedPrecision,
}

Quantization strategy

Variants

None

No quantization (FP32)

INT8WeightOnly

Quantize weights to INT8, keep activations in FP32

FP16

Quantize weights to FP16

BF16

Quantize weights to BF16

INT8Dynamic

Quantize both weights and activations to INT8 (dynamic)

MixedPrecision

Mixed precision: sensitive layers in FP32, others in INT8

Implementations

impl QuantStrategy

pub fn compression_ratio(&self) -> f32

Get memory compression ratio

pub fn quantizes_weights(&self) -> bool

Check if strategy quantizes weights

pub fn quantizes_activations(&self) -> bool

Check if strategy quantizes activations

Trait Implementations

impl Clone for QuantStrategy

fn clone(&self) -> QuantStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for QuantStrategy

fn eq(&self, other: &QuantStrategy) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for QuantStrategy

impl Eq for QuantStrategy

impl StructuralPartialEq for QuantStrategy