Struct DoubleQuantized4Bit 

pub struct DoubleQuantized4Bit {
    pub quantized_scales: Vec<u8>,
    pub super_scales: Vec<f32>,
    pub data: Vec<u8>,
    pub len: usize,
    pub num_blocks: usize,
}

Double-quantized 4-bit representation

Same 4-bit packed data as Quantized4Bit, but scale factors are stored as 8-bit unsigned values with a second-level FP32 super-scale per group of 256.

Fields

quantized_scales: Vec<u8>

8-bit quantized scale factors (one per first-level block)

super_scales: Vec<f32>

Super-scale factors (one FP32 per DOUBLE_QUANT_BLOCK_SIZE first-level blocks)

data: Vec<u8>

Quantized data: 2 values per byte (4 bits each) — identical to Quantized4Bit

len: usize

Original number of elements

num_blocks: usize

Number of first-level blocks

Implementations

impl DoubleQuantized4Bit

pub fn memory_bytes(&self) -> usize

Memory usage in bytes

pub fn compression_ratio(&self) -> f32

Compression ratio vs f32

pub fn double_quant_savings(&self) -> usize

Memory saved compared to single quantization (bytes)

Trait Implementations

impl Clone for DoubleQuantized4Bit

fn clone(&self) -> DoubleQuantized4Bit

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DoubleQuantized4Bit

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for DoubleQuantized4Bit

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for DoubleQuantized4Bit

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.