Skip to main content

Module vector

Module vector 

Source
Expand description

Vector indexing data structures

This module provides a modular architecture for vector search:

§Module Structure

  • ivf - Core IVF (Inverted File Index) infrastructure

    • CoarseCentroids - k-means clustering for coarse quantization
    • ClusterData / ClusterStorage - generic cluster storage
    • SoarConfig / MultiAssignment - SOAR geometry-aware assignment
  • quantization - Vector quantization methods

    • RaBitQCodebook - RaBitQ binary quantization (32x compression)
    • PQCodebook - Product Quantization with OPQ (ScaNN-style)
    • Quantizer trait - common interface for quantizers
  • index - Ready-to-use IVF indexes

    • IVFRaBitQIndex - IVF + RaBitQ
    • IVFPQIndex - IVF + PQ

§SOAR (Spilling with Orthogonality-Amplified Residuals)

The IVF module includes Google’s SOAR algorithm for improved recall:

  • Assigns vectors to multiple clusters (primary + secondary)
  • Secondary clusters chosen to have orthogonal residuals
  • Improves recall by 5-15% with ~1.3-2x storage overhead

Re-exports§

pub use ivf::ClusterData;
pub use ivf::ClusterStorage;
pub use ivf::CoarseCentroids;
pub use ivf::CoarseConfig;
pub use ivf::MultiAssignment;
pub use ivf::QuantizedCode;
pub use ivf::SoarConfig;
pub use quantization::DistanceTable;
pub use quantization::PQCodebook;
pub use quantization::PQConfig;
pub use quantization::PQVector;
pub use quantization::QuantizedQuery;
pub use quantization::QuantizedVector;
pub use quantization::Quantizer;
pub use quantization::RaBitQCodebook;
pub use quantization::RaBitQConfig;
pub use index::IVFPQConfig;
pub use index::IVFPQIndex;
pub use index::IVFRaBitQConfig;
pub use index::IVFRaBitQIndex;
pub use index::RaBitQIndex;

Modules§

index
Vector index implementations
ivf
IVF (Inverted File Index) module for vector search
quantization
Vector quantization methods for IVF indexes