Expand description
Optimised vector searches in Rust originally designed for single cell applications, but now as additionally GPU-accelerated, quantised (with binary indices) vector searches leveraging Rust’s performance under the hood.
§Feature flags
binary— Enables binary index support with optional memory-mapped files for on-disk vector storagegpu— Enables GPU-accelerated indices via the CubeCL backendgpu-tests— Enables GPU integration tests (requires ‘wgpu’ and a GPU on the machine)quantised— Enables quantised indices with different quantisations (BF16, SQ8, PQ, OPQ)mimalloc— Enables mimalloc as the global allocator for improved performance
Modules§
- binary
- This module contains all of the helpers, structures and methods related to binary indices. This includes the index structures themselves and distance calculations
- cpu
- Various CPU-based indices that run on the raw vectors. Leverage SIMD under the hood.
- prelude
- Re-exports of commonly used types and traits for convenient glob importing.
- quantised
- This module contains all of the helpers, structures and methods related to quantised indices (bf16, SQ8, PQ and OPQ).
- utils
- All types of utilities of this crate. From SIMD-accelerated distance calculations, to highly optimised k-means clustering to structure to be kept on the heap, shared traits, etd.
Functions§
- build_
annoy_ index - Build an Annoy index
- build_
balltree_ index - Build a BallTree index
- build_
exhaustive_ bf16_ index - Build an Exhaustive-BF16 index
- build_
exhaustive_ index - Build an exhaustive index
- build_
exhaustive_ index_ binary - Build an exhaustive binary index
- build_
exhaustive_ index_ rabitq - Build an exhaustive RaBitQ index
- build_
exhaustive_ opq_ index - Build an Exhaustive-OPQ index
- build_
exhaustive_ pq_ index - Build an Exhaustive-PQ index
- build_
exhaustive_ sq8_ index - Build an Exhaustive-SQ8 index
- build_
hnsw_ index - Build an HNSW index
- build_
ivf_ bf16_ index - Build an IVF-BF16 index
- build_
ivf_ index - Build an IVF index
- build_
ivf_ index_ binary - Build an IVF index with binary quantisation
- build_
ivf_ index_ rabitq - Build an IVF-RaBitQ index
- build_
ivf_ opq_ index - Build an IVF-OPQ index
- build_
ivf_ pq_ index - Build an IVF-PQ index
- build_
ivf_ sq8_ index - Build an IVF-SQ8 index
- build_
kd_ tree_ index - Build a Kd-Tree forest index
- build_
lsh_ index - Build the LSH index
- build_
nndescent_ index - Build an NNDescent index
- build_
vamana_ index - Build a Vamana index
- query_
annoy_ index - Helper function to query a given Annoy index
- query_
annoy_ self - Helper function to self query the Annoy index
- query_
balltree_ index - Helper function to query a given BallTree index
- query_
balltree_ self - Helper function to self query the BallTree index
- query_
exhaustive_ bf16_ index - Helper function to query a given Exhaustive-BF16 index
- query_
exhaustive_ bf16_ self - Helper function to self query a given Exhaustive-BF16 index
- query_
exhaustive_ index - Helper function to query a given exhaustive index
- query_
exhaustive_ index_ binary - Helper function to query a given exhaustive binary index
- query_
exhaustive_ index_ binary_ self - Query an exhaustive binary index against itself
- query_
exhaustive_ index_ rabitq - Helper function to query a given exhaustive RaBitQ index
- query_
exhaustive_ index_ rabitq_ self - Query an exhaustive RaBitQ index against itself
- query_
exhaustive_ opq_ index - Helper function to query a given Exhaustive-OPQ index
- query_
exhaustive_ opq_ index_ self - Helper function to self query an Exhaustive-OPQ index
- query_
exhaustive_ pq_ index - Helper function to query a given Exhaustive-PQ index
- query_
exhaustive_ pq_ index_ self - Helper function to self query an Exhaustive-PQ index
- query_
exhaustive_ self - Helper function to self query an exhaustive index
- query_
exhaustive_ sq8_ index - Helper function to query a given Exhaustive-SQ8 index
- query_
exhaustive_ sq8_ self - Helper function to self query a given Exhaustive-SQ8 index
- query_
hnsw_ index - Helper function to query a given HNSW index
- query_
hnsw_ self - Helper function to self query the HNSW index
- query_
ivf_ bf16_ index - Helper function to query a given IVF-BF16 index
- query_
ivf_ bf16_ self - Helper function to self query a given IVF-SQ8 index
- query_
ivf_ index - Helper function to query a given IVF index
- query_
ivf_ index_ binary - Query an IVF binary index
- query_
ivf_ index_ binary_ self - Query an IVF binary index against itself
- query_
ivf_ index_ rabitq - Helper function to query a given IVF-RaBitQ index
- query_
ivf_ index_ rabitq_ self - Query an IVF-RaBitQ index against itself
- query_
ivf_ opq_ index - Helper function to query a given IVF-OPQ index
- query_
ivf_ opq_ index_ self - Helper function to self query a IVF-OPQ index
- query_
ivf_ pq_ index - Helper function to query a given IVF-PQ index
- query_
ivf_ pq_ index_ self - Helper function to self query a IVF-PQ index
- query_
ivf_ self - Helper function to self query an IVF index
- query_
ivf_ sq8_ index - Helper function to query a given IVF-SQ8 index
- query_
ivf_ sq8_ self - Helper function to self query a given IVF-SQ8 index
- query_
kd_ tree_ index - Helper function to query a given Kd-Tree index
- query_
kd_ tree_ self - Helper function to self query the Kd-Tree index
- query_
lsh_ index - Helper function to query a given LSH index
- query_
lsh_ self - Helper function to self query an LSH index
- query_
nndescent_ index - Helper function to query a given NNDescent index
- query_
nndescent_ self - Helper function to self query the NNDescent index
- query_
vamana_ index - Query a Vamana index with an external query matrix
- query_
vamana_ self - Self-query a Vamana index to generate a full kNN graph