Crate faiss_next

Expand description

§faiss-next

Rust bindings for Faiss (Facebook AI Similarity Search), a library for efficient similarity search and clustering of dense vectors.

§Features

Safe Rust API: Idiomatic Rust wrappers around Faiss C API
Multiple Index Types: Support for Flat, IVF, LSH, Scalar Quantizer, and more
Multi-Version Support: Works with Faiss 1.14.x and newer (with loose compatibility mode)
CUDA Support: Optional GPU acceleration on Linux and Windows (feature flag: cuda)
Serialization: Save and load indexes to/from disk
Clustering: K-means clustering with customizable parameters
Pairwise Operations: Efficient distance computations (L2, inner product)

§Platform Support

OS	Architecture	CPU	CUDA
macOS (Apple Silicon)	aarch64 (M1/M2/M3)	✅	❌
Linux	x86_64	✅	✅
Windows	x86_64	✅	✅

Legend:

✅ Fully supported with pre-generated bindings
❌ Not supported

Notes:

CUDA is supported on Linux x86_64 and Windows x86_64

§Supported Index Types

Index Type	Description
`IndexFlat`	Brute-force index (exact search)
`IndexIVFFlat`	Inverted file with flat quantizer
`IndexIVFScalarQuantizer`	IVF with scalar quantization
`IndexScalarQuantizer`	Scalar quantizer index
`IndexLSH`	Locality-sensitive hashing
`IndexIDMap` / `IndexIDMap2`	Custom ID mapping wrapper
`IndexPreTransform`	Pre-transformation wrapper
`IndexRefineFlat`	Refinement with flat index
`IndexReplicas` / `IndexShards`	Distributed indexes
`IndexBinary`	Binary vector indexes
`IndexFlat1D`	Optimized 1D flat index

§Search Parameters

Type	Description
`SearchParameters`	Basic search parameters
`SearchParametersIvf`	IVF-specific parameters (`nprobe`, `max_codes`)

§Getting Started

§Prerequisites

Install Faiss C library:

macOS (Homebrew):

brew install faiss

Linux:

# From source (recommended)
git clone https://github.com/facebookresearch/faiss.git
cd faiss
mkdir build && cd build
cmake -DFAISS_ENABLE_C_API=ON -DBUILD_SHARED_LIBS=ON ..
make -j
sudo make install

Windows:

For Windows, you need to build Faiss from source with C API enabled. A pre-configured build is available at:

git clone -b windows-build https://github.com/yexiangyu/faiss.git
cd faiss
mkdir build && cd build
cmake -A x64 -DFAISS_ENABLE_C_API=ON -DBUILD_SHARED_LIBS=ON ^
      -DFAISS_ENABLE_GPU=ON ^
      -DCMAKE_INSTALL_PREFIX=C:/tools/faiss ..
cmake --build . --config Release
cmake --install . --config Release

After installation, set environment variables:

set FAISS_INCLUDE_DIR=C:\tools\faiss\include
set FAISS_LIB_DIR=C:\tools\faiss\lib

Or copy faiss.dll and faiss_c.dll to your executable directory.

Note: CUDA support on Windows requires:

NVIDIA CUDA Toolkit installed
Faiss built with -DFAISS_ENABLE_GPU=ON
MKL libraries (bundled with Intel oneAPI or conda)

Ensure the library is discoverable:
- Set FAISS_DIR environment variable, or
- Install to standard locations (/usr/local, /opt/homebrew)

§Installation

Add to your Cargo.toml:

[dependencies]
faiss-next = "0.6"

§Basic Usage

use faiss_next::{IndexFlat, Index};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create a flat L2 index
    let mut index = IndexFlat::new_l2(128)?;  // 128-dimensional vectors

    // Add vectors
    let vectors: Vec<f32> = vec![0.0; 128 * 100];  // 100 vectors of 128 dimensions
    index.add(&vectors)?;

    // Search for k nearest neighbors
    let query: Vec<f32> = vec![0.0; 128];
    let result = index.search(&query, 10)?;
    for i in 0..10 {
        println!("Label: {:?}, Distance: {}", result.labels[i], result.distances[i]);
    }
    Ok(())
}

§Using Index Factory

use faiss_next::{index_factory, MetricType, Index};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create IVF index
    let mut index = index_factory(128, "IVF256,Flat", MetricType::L2)?;

    // Train and add vectors
    let training_data: Vec<f32> = vec![0.0; 128 * 1000];
    index.train(&training_data)?;
    
    let vectors: Vec<f32> = vec![0.0; 128 * 100];
    index.add(&vectors)?;
    Ok(())
}

§Using Index Builder (Fluent API)

use faiss_next::{IndexBuilder, Index};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut index = IndexBuilder::new(128)
        .ivf_flat(256)         // IVF with 256 clusters
        .l2()
        .build()?;

    let training_data: Vec<f32> = vec![0.0; 128 * 1000];
    index.train(&training_data)?;
    
    let vectors: Vec<f32> = vec![0.0; 128 * 100];
    index.add(&vectors)?;
    Ok(())
}

§Custom IDs

use faiss_next::{index_factory, IndexIDMap, Index, MetricType};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let base = index_factory(128, "Flat", MetricType::L2)?;
    let mut index = IndexIDMap::new(base)?;

    let vectors: Vec<f32> = vec![0.0; 128 * 100];
    let ids: Vec<u64> = (100..200).collect();
    let ids_idx: Vec<faiss_next::Idx> = ids.iter().map(|&id| faiss_next::Idx::new(id)).collect();
    index.add_with_ids(&vectors, &ids_idx)?;
    Ok(())
}

§Serialization

use faiss_next::{IndexFlat, Index, write_index, read_index};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut index = IndexFlat::new_l2(128)?;
    let vectors: Vec<f32> = vec![0.0; 128 * 100];
    index.add(&vectors)?;

    // Save
    write_index(&index, "my_index.bin")?;

    // Load
    let loaded = read_index("my_index.bin")?;
    Ok(())
}

§Pairwise Distance Computation

use faiss_next::{pairwise_l2_sqr, inner_products};

let d = 128;
let x: Vec<f32> = vec![0.0; d * 10];  // 10 query vectors
let y: Vec<f32> = vec![0.0; d * 100]; // 100 database vectors

// Compute pairwise L2 squared distances
let distances = pairwise_l2_sqr(d, &x, &y);  // 10 * 100 elements

// Compute pairwise inner products
let products = inner_products(d, &x, &y);  // 10 * 100 elements

§Clustering (K-means)

use faiss_next::{Clustering, IndexFlat};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut clustering = Clustering::new(128, 100)?;  // d=128, k=100
    let mut index = IndexFlat::new_l2(128)?;

    let n = 1000u64;
    let data: Vec<f32> = vec![0.0; 128 * 1000];
    clustering.train(n, &data, &mut index)?;

    let centroids = clustering.centroids();
    Ok(())
}

§Search with Parameters

For fine-grained control over search behavior, use search_with_params:

use faiss_next::{index_factory, MetricType, Index, SearchParameters, SearchParametersIvf};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut index = index_factory(128, "IVF256,Flat", MetricType::L2)?;
    
    let training_data: Vec<f32> = vec![0.0; 128 * 1000];
    index.train(&training_data)?;
    index.add(&training_data)?;

    let query: Vec<f32> = vec![0.0; 128];
    
    // Basic search parameters
    let params = SearchParameters::new()?;
    let result = index.search_with_params(&query, 10, &params)?;

    // IVF-specific parameters (nprobe, max_codes)
    let mut ivf_params = SearchParametersIvf::new()?;
    ivf_params.set_nprobe(16);       // Search 16 clusters
    ivf_params.set_max_codes(10000); // Max codes to visit
    let result = index.search_with_params(&query, 10, &ivf_params)?;
    Ok(())
}

§Range Search

Find all vectors within a distance threshold:

use faiss_next::{IndexFlat, Index};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut index = IndexFlat::new_l2(128)?;
    let vectors: Vec<f32> = vec![0.0; 128 * 100];
    index.add(&vectors)?;

    let query: Vec<f32> = vec![0.0; 128];
    
    // Find all vectors within radius 10.0
    let result = index.range_search(&query, 10.0)?;

    // Iterate over results
    for (labels, distances) in result.iter() {
        println!("Found {} results", labels.len());
    }

    // Get results for a specific query
    if let Some((labels, distances)) = result.get(0) {
        println!("Query 0: {} results within radius", labels.len());
    }
    Ok(())
}

§Feature Flags

Flag	Description
`cuda`	Enable CUDA GPU support (Linux x86_64 and Windows x86_64)
`bindgen`	Generate bindings at compile time (requires LLVM/Clang)

§Performance

The bindings leverage Faiss’s optimized SIMD implementations:

L2 Distance Benchmark (Dimension: 128)
Size (nq, nb)      ndarray (ms)    faiss (ms)     Speedup
(  100,  1000)           12.34          0.87        14.2x
( 1000,  1000)          123.45          8.21        15.0x
( 1000, 10000)         1234.56         17.89        69.0x

§Version Compatibility

Minimum: Faiss 1.14.0
Tested: Faiss 1.14.x
Loose Mode: Newer versions will work with a compile-time warning

When Faiss 1.15 is released, add new bindings in the v1_15 directory.

§Documentation

§License

MIT License

§Acknowledgments

Faiss by Facebook AI Research
Inspired by faiss-rs

Re-exports§

pub use clustering::Clustering;
pub use clustering::ClusteringParameters;
pub use distance::DistanceComputer;
pub use error::Error;
pub use error::Result;
pub use factory::index_factory;
pub use factory::IndexBuilder;
pub use id_selector::IDSelector;
pub use id_selector::IDSelectorAnd;
pub use id_selector::IDSelectorBatch;
pub use id_selector::IDSelectorNot;
pub use id_selector::IDSelectorOr;
pub use id_selector::IDSelectorRange;
pub use id_selector::IDSelectorXOr;
pub use idx::Idx;
pub use index::BinaryIndex;
pub use index::Index;
pub use index::IndexBinary;
pub use index::IndexFlat;
pub use index::IndexFlat1D;
pub use index::IndexIDMap;
pub use index::IndexIDMap2;
pub use index::IndexIVF;
pub use index::IndexIVFFlat;
pub use index::IndexIVFScalarQuantizer;
pub use index::IndexImpl;
pub use index::IndexLSH;
pub use index::IndexPreTransform;
pub use index::IndexRefineFlat;
pub use index::IndexReplicas;
pub use index::IndexScalarQuantizer;
pub use index::IndexShards;
pub use index::IvfIndex;
pub use index::QuantizerType;
pub use io::read_index;
pub use io::read_index_binary;
pub use io::write_index;
pub use io::write_index_binary;
pub use metric::MetricType;
pub use pairwise::get_distance_compute_blas_database_bs;
pub use pairwise::get_distance_compute_blas_query_bs;
pub use pairwise::get_distance_compute_blas_threshold;
pub use pairwise::inner_products;
pub use pairwise::l2_sqr_ny;
pub use pairwise::norm_l2_sqr;
pub use pairwise::norms_l2;
pub use pairwise::norms_l2_sqr;
pub use pairwise::pairwise_l2_sqr;
pub use pairwise::pairwise_l2_sqr_with_stride;
pub use pairwise::renorm_l2;
pub use pairwise::set_distance_compute_blas_database_bs;
pub use pairwise::set_distance_compute_blas_query_bs;
pub use pairwise::set_distance_compute_blas_threshold;
pub use parameter::ParameterSpace;
pub use result::BinarySearchResult;
pub use result::RangeSearchResult;
pub use result::SearchResult;
pub use search_params::SearchParameters;
pub use search_params::SearchParametersIvf;
pub use search_params::SearchParams;
pub use transform::CenteringTransform;
pub use transform::ItqMatrix;
pub use transform::ItqTransform;
pub use transform::LinearTransform;
pub use transform::NormalizationTransform;
pub use transform::OpqMatrix;
pub use transform::PcaMatrix;
pub use transform::RandomRotationMatrix;
pub use transform::RemapDimensionsTransform;
pub use transform::VectorTransform;

Modules§

clustering
distance
error
factory
id_selector
idx
index
io
metric
pairwise
parameter
result
search_params
transform

Crate faiss_next

Crate faiss_next Copy item path

§faiss-next

§Features

§Platform Support

§Supported Index Types

§Search Parameters

§Getting Started

§Prerequisites

§Installation

§Basic Usage

§Using Index Factory

§Using Index Builder (Fluent API)

§Custom IDs

§Serialization

§Pairwise Distance Computation

§Clustering (K-means)

§Search with Parameters

§Range Search

§Feature Flags

§Performance

§Version Compatibility

§Documentation

§License

§Acknowledgments

Re-exports§

Modules§

Crate faiss_next