Enum QuantMethod 

pub enum QuantMethod {
    Q4_0,
    Q4KM,
    Q5KM,
    Q8_0,
    F16,
    F32,
}

Supported quantization methods.

Variants

Q4_0

4-bit quantization (fastest inference, lowest quality).

Q4KM

4-bit K-quant medium (good balance).

Q5KM

5-bit K-quant medium.

Q8_0

8-bit quantization (highest quality quantized).

F16

16-bit float (half precision).

F32

32-bit float (full precision, no quantization).

Implementations

impl QuantMethod

pub fn bits_per_weight(self) -> f64

Approximate bits-per-weight for this method.

pub fn as_conversion_str(&self) -> &'static str

Returns the string used in conversion options.

Trait Implementations

impl Clone for QuantMethod

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QuantMethod

impl Debug for QuantMethod

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for QuantMethod

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for QuantMethod

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for QuantMethod

impl FromStr for QuantMethod

type Err = VaultError

The associated error which can be returned from parsing.

fn from_str(s: &str) -> Result<Self, Self::Err>

Parses a string s to return a value of this type.

impl PartialEq for QuantMethod

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for QuantMethod

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for QuantMethod