Struct VectorIndex 

pub struct VectorIndex { /* private fields */ }

HNSW-based vector index for semantic search

Provides efficient approximate k-nearest neighbor search over high-dimensional vectors associated with content IDs.

Implementations

impl VectorIndex

pub fn new( dimension: usize, metric: DistanceMetric, max_nb_connection: usize, ef_construction: usize, ) -> Result<Self>

Create a new vector index with the specified dimension

Arguments

• dimension - Dimension of vectors to be indexed
• metric - Distance metric to use
• max_nb_connection - Maximum number of connections per layer (M parameter)
• ef_construction - Size of dynamic candidate list (efConstruction parameter)

pub fn with_defaults(dimension: usize) -> Result<Self>

Create a new index with default parameters

Uses M=16 and efConstruction=200, which are good defaults for most use cases

pub fn insert(&mut self, cid: &Cid, vector: &[f32]) -> Result<()>

Insert a vector associated with a CID

Arguments

• cid - Content identifier
• vector - Feature vector to index

pub fn search( &self, query: &[f32], k: usize, ef_search: usize, ) -> Result<Vec<SearchResult>>

Search for k nearest neighbors

Arguments

• query - Query vector
• k - Number of neighbors to return
• ef_search - Size of dynamic candidate list during search (higher = more accurate but slower)

pub fn delete(&mut self, cid: &Cid) -> Result<()>

Delete a vector by CID

pub fn contains(&self, cid: &Cid) -> bool

Check if a CID exists in the index

pub fn len(&self) -> usize

Get the number of vectors in the index

pub fn is_empty(&self) -> bool

Check if the index is empty

pub fn dimension(&self) -> usize

Get the dimension of vectors in this index

pub fn metric(&self) -> DistanceMetric

Get the distance metric used by this index

pub fn get_all_cids(&self) -> Vec<Cid>

Get all CIDs in the index Useful for synchronization and snapshots

pub fn get_embedding(&self, cid: &Cid) -> Option<Vec<f32>>

Get the embedding vector for a specific CID

Returns None if the CID is not in the index

pub fn get_all_embeddings(&self) -> Vec<(Cid, Vec<f32>)>

Get all embeddings in the index as (CID, vector) pairs

Useful for iteration, migration, and batch operations

pub fn iter(&self) -> Vec<(Cid, Vec<f32>)>

Iterate over all (CID, vector) pairs in the index

Returns an iterator over the embeddings

pub fn compute_optimal_parameters(&self) -> (usize, usize)

Compute optimal HNSW parameters based on current index size

Returns recommended (max_nb_connection, ef_construction) based on:

• Small indexes (< 10k): M=16, ef=200
• Medium indexes (10k-100k): M=32, ef=400
• Large indexes (> 100k): M=48, ef=600

pub fn compute_optimal_ef_search(&self, k: usize) -> usize

Get recommended ef_search parameter based on k

Generally ef_search should be >= k and higher for better recall

pub fn get_parameter_recommendations( &self, use_case: UseCase, ) -> ParameterRecommendation

Get detailed parameter recommendations based on use case

pub fn insert_batch(&mut self, items: &[(Cid, Vec<f32>)]) -> Result<()>

Insert multiple vectors in batch

More efficient than inserting one by one as it can use parallelization

Arguments

• items - Vector of (CID, vector) pairs to insert

pub fn insert_incremental( &mut self, items: &[(Cid, Vec<f32>)], chunk_size: usize, ) -> Result<IncrementalBuildStats>

Insert vectors incrementally with periodic optimization

This method inserts vectors in chunks and tracks statistics to determine if index rebuild is beneficial. Returns statistics about the insertion.

Arguments

• items - Vector of (CID, vector) pairs to insert
• chunk_size - Number of vectors to insert before checking optimization

Returns

Statistics about the incremental build process

pub fn should_rebuild(&self) -> bool

Determine if index should be rebuilt for better performance

Rebuild is recommended when:

• Index has grown significantly (2x or more)
• Many deletions have occurred (fragmentation)
• Current parameters are suboptimal for index size

pub fn rebuild(&mut self, use_case: UseCase) -> Result<RebuildStats>

Rebuild the index with optimal parameters for current size

This creates a new index with better parameters and re-inserts all vectors. Use this when should_rebuild() returns true.

Arguments

• use_case - Target use case for parameter selection

pub fn get_build_stats(&self) -> BuildHealthStats

Get statistics about incremental build performance

pub fn save(&self, path: impl AsRef<Path>) -> Result<()>

Save the index to a file

Saves the HNSW index and CID mappings to disk for later retrieval. The index is saved in oxicode format.

Arguments

• path - Path to save the index to

pub fn load(path: impl AsRef<Path>) -> Result<Self>

Load an index from a file

Loads a previously saved index from disk.

Arguments

• path - Path to load the index from