Skip to main content

Crate embeddenator_retrieval

Crate embeddenator_retrieval 

Source
Expand description

§embeddenator-retrieval

Semantic retrieval and search operations for VSA-based vector representations.

This crate provides:

  • Inverted indexing for fast approximate search
  • Multiple similarity metrics (cosine, Hamming, Jaccard)
  • Search strategies (exact, approximate, two-stage, hierarchical)
  • Index structures (brute force, hierarchical)
  • Resonator networks for pattern completion and factorization
  • Algebraic correction for guaranteed reconstruction

Extracted from embeddenator core as part of Phase 2A component decomposition. See ADR-016.

§Examples

§Basic Retrieval

use embeddenator_retrieval::{TernaryInvertedIndex, search::two_stage_search, search::SearchConfig};
use embeddenator_vsa::SparseVec;
use std::collections::HashMap;

// Build index
let mut index = TernaryInvertedIndex::new();
let mut vectors = HashMap::new();

let vec1 = SparseVec::from_data(b"document one");
let vec2 = SparseVec::from_data(b"document two");

index.add(1, &vec1);
index.add(2, &vec2);
index.finalize();

vectors.insert(1, vec1);
vectors.insert(2, vec2);

// Search
let query = SparseVec::from_data(b"document");
let config = SearchConfig::default();
let results = two_stage_search(&query, &index, &vectors, &config, 5);

assert!(!results.is_empty());

§Similarity Metrics

use embeddenator_retrieval::similarity::{compute_similarity, SimilarityMetric};
use embeddenator_vsa::SparseVec;

let a = SparseVec::from_data(b"hello");
let b = SparseVec::from_data(b"hello");

let cosine = compute_similarity(&a, &b, SimilarityMetric::Cosine);
let hamming = compute_similarity(&a, &b, SimilarityMetric::Hamming);
let jaccard = compute_similarity(&a, &b, SimilarityMetric::Jaccard);

assert!(cosine > 0.9);
assert!(hamming < 10.0);

Re-exports§

pub use core::correction;
pub use core::resonator;
pub use hnsw::HNSWConfig;
pub use hnsw::HNSWIndex;
pub use hnsw::HNSWStats;
pub use index::BruteForceIndex;
pub use index::HierarchicalIndex;
pub use index::IndexConfig;
pub use index::RetrievalIndex;
pub use search::exact_search_parallel;
pub use search::RankedResult;
pub use search::SearchConfig;
pub use similarity::compute_similarity;
pub use similarity::SimilarityMetric;
pub use distributed::DistributedConfig;
pub use distributed::DistributedError;
pub use distributed::DistributedResult;
pub use distributed::DistributedSearch;
pub use distributed::DistributedSearchBuilder;
pub use distributed::QueryStats;
pub use distributed::Shard;
pub use distributed::ShardAssigner;
pub use distributed::ShardId;
pub use distributed::ShardResult;
pub use distributed::ShardStatus;
pub use distributed::ShardingStrategy;
pub use retrieval::*;

Modules§

core
distributed
Distributed Search Infrastructure
hnsw
Hierarchical Navigable Small World (HNSW) Index
index
Index structures for efficient retrieval
retrieval
search
Search strategies for semantic retrieval
similarity
Similarity metrics for VSA-based retrieval

Structs§

IndexBuilder
Builder for creating a search index
QueryEngine
Query engine for search operations
QueryResult
Query result
SearchIndex
Search index for querying