Struct BitsAndBytesConfig 

pub struct BitsAndBytesConfig {
    pub bits: u8,
    pub dynamic_tree: bool,
    pub block_size: usize,
    pub stochastic: bool,
    pub outlier_threshold: f32,
    pub nested_quantization: bool,
}

Quantization configuration compatible with bitsandbytes

Fields

bits: u8

Quantization bit width (4 or 8)

dynamic_tree: bool

Use dynamic tree quantization

block_size: usize

Block size for block-wise quantization

stochastic: bool

Use stochastic quantization

outlier_threshold: f32

Percentile for outlier detection

nested_quantization: bool

Use nested quantization for scales

Trait Implementations

impl Clone for BitsAndBytesConfig

fn clone(&self) -> BitsAndBytesConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BitsAndBytesConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for BitsAndBytesConfig

fn default() -> Self

Returns the “default value” for a type.