Expand description
Scalar Int8 vector quantization for ~4x memory reduction vs f32.
Each vector is quantized per-dimension within the bounds of a calibration set (min/max per dimension derived from a sample of vectors). Dequantized dot products approximate the originals and are suitable for an initial ANN coarse pass before an exact rerank.
Structsยง
- Quantization
Params - Per-dimension quantization calibration: lower and upper bounds.
- Quantized
Index - A quantized vector store. Lookup is by id; search returns top-k by approximate dot product (dequantized), preserving the order of insertion.