Struct QuantizedTensor 

pub struct QuantizedTensor {
    pub data: Vec<u8>,
    pub scales: Vec<f32>,
    pub zero_points: Option<Vec<f32>>,
    pub scales_quantized: Option<Vec<u8>>,
    pub scales_scales: Option<Vec<f32>>,
    pub shape: Vec<usize>,
    pub block_size: usize,
    pub double_quant_enabled: bool,
    pub strategy: QuantizationStrategy,
}

A quantized tensor with scale factors.

Fields

data: Vec<u8>

Packed 4-bit values (2 values per byte).

scales: Vec<f32>

Scale factors per block.

zero_points: Option<Vec<f32>>

Zero points per block (for asymmetric quantization).

scales_quantized: Option<Vec<u8>>

Quantized scales (when double quantization is used).

scales_scales: Option<Vec<f32>>

Scale factors for the scales (when double quantization is used).

shape: Vec<usize>

Original shape.

block_size: usize

Block size used for quantization.

double_quant_enabled: bool

Whether double quantization was applied.

strategy: QuantizationStrategy

Quantization strategy used.

Implementations

impl QuantizedTensor

pub fn numel(&self) -> usize

Get the number of elements in the original tensor.

pub fn size_bytes(&self) -> usize

Get the memory size in bytes.

pub fn compression_ratio(&self) -> f64

Get the compression ratio relative to FP32 format.

Trait Implementations

impl Debug for QuantizedTensor

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.