Expand description
Vector indexing data structures
This module provides a modular architecture for vector search:
§Module Structure
-
ivf- Core IVF (Inverted File Index) infrastructureCoarseCentroids- k-means clustering for coarse quantizationClusterData/ClusterStorage- generic cluster storageSoarConfig/MultiAssignment- SOAR geometry-aware assignment
-
quantization- Vector quantization methodsRaBitQCodebook- RaBitQ binary quantization (32x compression)PQCodebook- Product Quantization with OPQ (ScaNN-style)Quantizertrait - common interface for quantizers
-
index- Ready-to-use IVF indexesIVFRaBitQIndex- IVF + RaBitQIVFPQIndex- IVF + PQ
§SOAR (Spilling with Orthogonality-Amplified Residuals)
The IVF module includes Google’s SOAR algorithm for improved recall:
- Assigns vectors to multiple clusters (primary + secondary)
- Secondary clusters chosen to have orthogonal residuals
- Improves recall by 5-15% with ~1.3-2x storage overhead
Re-exports§
pub use ivf::ClusterData;pub use ivf::ClusterStorage;pub use ivf::CoarseCentroids;pub use ivf::CoarseConfig;pub use ivf::MultiAssignment;pub use ivf::QuantizedCode;pub use ivf::SoarConfig;pub use quantization::DistanceTable;pub use quantization::PQCodebook;pub use quantization::PQConfig;pub use quantization::PQVector;pub use quantization::QuantizedQuery;pub use quantization::QuantizedVector;pub use quantization::Quantizer;pub use quantization::RaBitQCodebook;pub use quantization::RaBitQConfig;pub use index::IVFPQConfig;pub use index::IVFPQIndex;pub use index::IVFRaBitQConfig;pub use index::IVFRaBitQIndex;pub use index::RaBitQIndex;
Modules§
- index
- Vector index implementations
- ivf
- IVF (Inverted File Index) module for vector search
- quantization
- Vector quantization methods for IVF indexes