Struct EmbeddingQuantizer 

pub struct EmbeddingQuantizer {
    pub bits: u8,
}

Batch quantizer supporting 4-bit and 8-bit precision.

Fields

bits: u8

Quantization bit width (4 or 8).

Implementations

impl EmbeddingQuantizer

pub fn new(bits: u8) -> Self

Create a new quantizer with the specified bit width.

bits should be 4 or 8; other values are accepted but treated as 8.

pub fn quantize_batch(&self, embeddings: &[Vec<f32>]) -> Vec<QuantizedEmbedding>

Quantize a batch of embeddings.

pub fn dequantize_batch( &self, quantized: &[QuantizedEmbedding], ) -> Vec<Vec<f32>>

Dequantize a batch of quantized embeddings.

pub fn compression_ratio(&self, original: &[Vec<f32>]) -> f64

Compute compression ratio: original_bytes / quantized_bytes.

Trait Implementations

impl Clone for EmbeddingQuantizer

fn clone(&self) -> EmbeddingQuantizer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EmbeddingQuantizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.