Expand description
Distributed vector search with sharding and replication.
This module provides distributed search capabilities for scaling to billions of vectors:
- Horizontal Sharding: Split vectors across multiple nodes
- Consistent Hashing: Load balancing for shard assignments
- Query Routing: Fan-out queries to all relevant shards
- Result Merging: Combine and re-rank results from multiple shards
- Replication: Fault tolerance with configurable replica count
§Example
use oxify_vector::distributed::{DistributedIndex, ShardConfig, ConsistentHash};
use oxify_vector::{SearchConfig, DistanceMetric};
use std::collections::HashMap;
// Configure distributed index with 3 shards
let shard_config = ShardConfig::new(3, 2); // 3 shards, 2 replicas
let search_config = SearchConfig::default();
let mut index = DistributedIndex::new(shard_config, search_config);
// Build index with automatic sharding
let mut embeddings = HashMap::new();
embeddings.insert("doc1".to_string(), vec![0.1, 0.2, 0.3]);
embeddings.insert("doc2".to_string(), vec![0.2, 0.3, 0.4]);
index.build(&embeddings)?;
// Search across all shards
let query = vec![0.15, 0.25, 0.35];
let results = index.search(&query, 5)?;Structs§
- Consistent
Hash - Consistent hashing for load balancing across shards.
- Distributed
Index - Distributed vector search index with sharding and replication.
- Distributed
Stats - Statistics for a distributed index.
- Shard
Config - Configuration for distributed sharding.