Enum QuantType 

pub enum QuantType {
    Q8_0,
    Q4_0,
    Q4_1,
    Q5_0,
    Q5_1,
    F16,
    F32,
}

Quantization type enumeration.

Variants

Q8_0

8-bit quantization with per-block scale. Format: scale (f16) + 32 x int8

Q4_0

4-bit quantization with per-block scale. Format: scale (f16) + 16 x uint8 (two 4-bit values each)

Q4_1

4-bit quantization with per-block scale and min. Format: scale (f16) + min (f16) + 16 x uint8

Q5_0

5-bit quantization with per-block scale.

Q5_1

5-bit quantization with per-block scale and min.

F16

Half-precision (16-bit float).

F32

Full precision (32-bit float).

Implementations

impl QuantType

pub fn block_size(&self) -> usize

Returns the block size for this quantization type.

pub fn bytes_per_block(&self) -> usize

Returns the number of bytes per block.

pub fn bits_per_value(&self) -> usize

Returns the bits per value.

pub fn compression_ratio(&self) -> f32

Returns the compression ratio compared to F32.

pub fn is_block_quantized(&self) -> bool

Returns true if this type uses block quantization.

pub fn from_str(s: &str) -> Option<Self>

Parses a quantization type from a string.

Trait Implementations

impl Clone for QuantType

fn clone(&self) -> QuantType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for QuantType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for QuantType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for QuantType

fn eq(&self, other: &QuantType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for QuantType

impl Eq for QuantType

impl StructuralPartialEq for QuantType