Expand description
Vector indexing data structures
This module provides a modular architecture for vector search:
§Module Structure
-
ivf- Core IVF (Inverted File Index) infrastructureCoarseCentroids- k-means clustering for coarse quantizationSoarConfig/MultiAssignment- SOAR geometry-aware assignment
-
quantization- TurboQuant training-free codecTqCodec- derived rotation/codebook, no trained artifacts
-
index- Segment payloads for the production ANN implementationsIvfTqIndex- float vectors with TQ-coded centroid residualsBinaryIvfIndex- exact packed binary vectors
§SOAR (Spilling with Orthogonality-Amplified Residuals)
The IVF module includes Google’s SOAR algorithm for improved recall:
- Assigns vectors to multiple clusters (primary + secondary)
- Secondary clusters chosen to have orthogonal residuals
- Improves recall by 5-15% with ~1.3-2x storage overhead
Re-exports§
pub use ivf::CoarseCentroids;pub use ivf::CoarseConfig;pub use ivf::IvfProbePlan;pub use ivf::MultiAssignment;pub use ivf::SoarConfig;pub use quantization::TqFlatBuilder;pub use quantization::TqCodec;pub use quantization::TqQueryPlan;pub use index::BinaryCoarseQuantizer;pub use index::BinaryIvfConfig;pub use index::BinaryIvfIndex;pub use index::IvfTqIndex;pub use index::TqIvfEncodeScratch;pub use index::TqIvfQueryPlan;pub use index::is_ivf_tq_cosine_generation;pub use index::mark_ivf_tq_cosine_generation;
Modules§
- index
- Vector index implementations
- ivf
- IVF (Inverted File Index) module for vector search
- quantization
- Vector quantization for IVF indexes