Skip to main content

Crate mnemonist_quant

Crate mnemonist_quant 

Source
Expand description

TurboQuant: online vector quantization with near-optimal distortion rate.

Implements the algorithms from TurboQuant (arXiv:2504.19874):

  • TurboQuantMse — MSE-optimal quantizer using random rotation + Lloyd-Max codebooks
  • TurboQuantProd — unbiased inner-product quantizer (MSE + QJL residual)
  • CompressedEmbeddingStore — binary storage format for quantized embeddings

Re-exports§

pub use codebook::Codebook;
pub use mse::QuantizedVector;
pub use mse::TurboQuantMse;
pub use prod::QuantizedProdVector;
pub use prod::TurboQuantProd;
pub use qjl::QjlTransform;
pub use rotation::Rotation;
pub use store::CompressedEmbeddingStore;

Modules§

codebook
Precomputed Lloyd-Max optimal scalar quantizer codebooks.
mse
TurboQuant_mse: MSE-optimal vector quantizer (Algorithm 1).
pack
Bit-packing utilities for storing b-bit indices compactly.
prod
TurboQuant_prod: unbiased inner-product quantizer (Algorithm 2).
qjl
Quantized Johnson-Lindenstrauss (QJL) 1-bit transform.
rotation
Random orthogonal rotation matrix for TurboQuant.
store
Compressed embedding store using TurboQuant quantization.

Enums§

QuantError
Errors specific to quantization operations.