Struct VectorIndex 

pub struct VectorIndex { /* private fields */ }

HNSW-based vector index for semantic search

Provides efficient approximate k-nearest neighbor search over high-dimensional vectors associated with content IDs.

Implementations

impl VectorIndex

pub fn new( dimension: usize, metric: DistanceMetric, max_nb_connection: usize, ef_construction: usize, ) -> Result<Self>

Create a new vector index with the specified dimension

Arguments

• dimension - Dimension of vectors to be indexed
• metric - Distance metric to use
• max_nb_connection - Maximum number of connections per layer (M parameter)
• ef_construction - Size of dynamic candidate list (efConstruction parameter)

pub fn with_defaults(dimension: usize) -> Result<Self>

Create a new index with default parameters

Uses M=16 and efConstruction=200, which are good defaults for most use cases

pub fn insert(&mut self, cid: &Cid, vector: &[f32]) -> Result<()>

Insert a vector associated with a CID

Arguments

• cid - Content identifier
• vector - Feature vector to index

pub fn add_embedding(&mut self, cid: &Cid, vector: &[f32]) -> Result<()>

Add an embedding for a CID — ergonomic alias for insert.

Marks the entry as dirty in the incremental tracker so that IndexPersistence can decide whether a full or incremental snapshot should be written next time it is called.

pub fn search( &self, query: &[f32], k: usize, ef_search: usize, ) -> Result<Vec<SearchResult>>

Search for k nearest neighbors

Arguments

• query - Query vector
• k - Number of neighbors to return
• ef_search - Size of dynamic candidate list during search (higher = more accurate but slower)

pub fn delete(&mut self, cid: &Cid) -> Result<()>

Delete a vector by CID

pub fn contains(&self, cid: &Cid) -> bool

Check if a CID exists in the index

pub fn len(&self) -> usize

Get the number of vectors in the index

pub fn is_empty(&self) -> bool

Check if the index is empty

pub fn dimension(&self) -> usize

Get the dimension of vectors in this index

pub fn metric(&self) -> DistanceMetric

Get the distance metric used by this index

pub fn get_all_cids(&self) -> Vec<Cid>

Get all CIDs in the index Useful for synchronization and snapshots

pub fn get_embedding(&self, cid: &Cid) -> Option<Vec<f32>>

Get the embedding vector for a specific CID

Returns None if the CID is not in the index

pub fn get_all_embeddings(&self) -> Vec<(Cid, Vec<f32>)>

Get all embeddings in the index as (CID, vector) pairs

Useful for iteration, migration, and batch operations

pub fn iter(&self) -> Vec<(Cid, Vec<f32>)>

Iterate over all (CID, vector) pairs in the index

Returns an iterator over the embeddings

pub fn estimated_memory_bytes(&self) -> usize

Estimated memory usage in bytes for the current index.

Approximation based on:

• Each node stores a float32 vector: dim * 4 bytes
• Each node stores neighbour pointers (2 per connection): m * 8 bytes

The HNSW max_nb_connection (m) is read from the underlying index so the estimate tracks the actual build parameters.

pub fn compute_optimal_parameters(&self) -> (usize, usize)

Compute optimal HNSW parameters based on current index size

Returns recommended (max_nb_connection, ef_construction) based on:

• Small indexes (< 10k): M=16, ef=200
• Medium indexes (10k-100k): M=32, ef=400
• Large indexes (> 100k): M=48, ef=600

pub fn compute_optimal_ef_search(&self, k: usize) -> usize

Get recommended ef_search parameter based on k

Generally ef_search should be >= k and higher for better recall

pub fn get_parameter_recommendations( &self, use_case: UseCase, ) -> ParameterRecommendation

Get detailed parameter recommendations based on use case

pub fn insert_batch(&mut self, items: &[(Cid, Vec<f32>)]) -> Result<()>

Insert multiple vectors in batch

More efficient than inserting one by one as it can use parallelization

Arguments

• items - Vector of (CID, vector) pairs to insert

pub fn insert_incremental( &mut self, items: &[(Cid, Vec<f32>)], chunk_size: usize, ) -> Result<IncrementalBuildStats>

Insert vectors incrementally with periodic optimization

This method inserts vectors in chunks and tracks statistics to determine if index rebuild is beneficial. Returns statistics about the insertion.

Arguments

• items - Vector of (CID, vector) pairs to insert
• chunk_size - Number of vectors to insert before checking optimization

Returns

Statistics about the incremental build process

pub fn should_rebuild(&self) -> bool

Determine if index should be rebuilt for better performance

Rebuild is recommended when:

• Index has grown significantly (2x or more)
• Many deletions have occurred (fragmentation)
• Current parameters are suboptimal for index size

pub fn rebuild(&mut self, use_case: UseCase) -> Result<RebuildStats>

Rebuild the index with optimal parameters for current size

This creates a new index with better parameters and re-inserts all vectors. Use this when should_rebuild() returns true.

Arguments

• use_case - Target use case for parameter selection

pub fn get_build_stats(&self) -> BuildHealthStats

Get statistics about incremental build performance

pub fn save(&self, path: impl AsRef<Path>) -> Result<()>

Save the index to a file

Saves the HNSW index and CID mappings to disk for later retrieval. The index is saved in oxicode format.

Arguments

• path - Path to save the index to

pub fn load(path: impl AsRef<Path>) -> Result<Self>

Load an index from a file

Loads a previously saved index from disk.

Arguments

• path - Path to load the index from

pub fn snapshot(&self) -> Result<IndexSnapshot>

Export the current index state as a portable crate::persistence::IndexSnapshot

The snapshot captures every vector and its CID mapping. Graph topology (layer connections) is approximated from stored metadata; the hnsw_rs crate does not expose raw adjacency lists, so on reload the graph is rebuilt by re-inserting all vectors in their original order.

Errors

Returns an error if any internal lock is poisoned.

pub fn snapshot_incremental( &self, base_version: u64, ) -> Result<IncrementalSnapshot>

Build an IncrementalSnapshot containing only the entries that have been inserted or modified since the last full or incremental snapshot.

The caller should call mark_tracker_clean after successfully persisting the returned snapshot.

Errors

Returns an error if any internal lock is poisoned.

pub fn from_snapshot(snapshot: &IndexSnapshot) -> Result<Self>

Restore a VectorIndex from a previously taken crate::persistence::IndexSnapshot

All vectors are re-inserted into a freshly created HNSW graph so the graph topology is fully rebuilt. The distance metric stored in the snapshot’s metadata_map under the key "metric" is used when present; otherwise L2 is assumed.

Errors

Returns an error if any entry has a vector with the wrong dimension, or if a CID string cannot be parsed.

pub fn dirty_count(&self) -> usize

Return the number of dirty (unsaved) entries tracked since the last snapshot.

pub fn tracker_version(&self) -> u64

Return the current incremental tracker version.

pub fn mark_tracker_clean(&self)

Mark the tracker as clean (call after a successful snapshot save).

pub fn record_full_snapshot(&self)

Record a full snapshot was taken now (resets dirty set and advances version).