Crate vecstore 

vecstore

A lightweight, high-performance vector database for Rust applications.

vecstore is designed to be the “SQLite of vector search” - an embeddable, easy-to-use vector store with HNSW indexing, metadata filtering, and persistence.

Features

• Fast approximate nearest neighbor search using HNSW (Hierarchical Navigable Small World) graphs
• Metadata filtering with SQL-like syntax ("category = 'tech' AND score > 0.5")
• Hybrid search combining vector similarity + BM25 keyword matching
• Product Quantization for 8-32x memory compression
• Persistence with snapshot/backup support
• Async API for Tokio applications
• Python bindings via PyO3
• WebAssembly support for browser usage
• Built-in embeddings via ONNX Runtime

Quick Start

use vecstore::{VecStore, Query};

// Create or open a vector store
let mut store = VecStore::open("my_vectors.db")?;

// Insert vectors with metadata
store.upsert(
    "doc1",
    vec![0.1, 0.2, 0.3, 0.4],
    serde_json::json!({
        "title": "First Document",
        "category": "tech",
        "score": 0.95
    }),
)?;

// Search with filters
let query = Query::new(vec![0.15, 0.25, 0.35, 0.45])
    .with_limit(10)
    .with_filter("category = 'tech' AND score > 0.9");

let results = store.query(query)?;

for result in results {
    println!("{}: {:.4}", result.id, result.distance);
}

Examples

Hybrid Search (Vector + Keyword)

use vecstore::{VecStore, HybridQuery};

let mut store = VecStore::open("vectors.db")?;

// Index text for keyword search
store.index_text("doc1", "machine learning tutorial")?;

// Hybrid search: 70% vector similarity + 30% keyword match
let query = HybridQuery::new(
    vec![0.1, 0.2, 0.3],
    "machine learning",
)
.with_limit(10)
.with_alpha(0.7);

let results = store.hybrid_query(query)?;

Product Quantization (Memory Compression)

use vecstore::{ProductQuantizer, PQConfig};

let config = PQConfig {
    num_subvectors: 16,
    num_centroids: 256,
    training_iterations: 20,
};

let mut pq = ProductQuantizer::new(128, config)?;

// Train on representative sample
pq.train(&training_vectors)?;

// Encode vectors (128 floats -> 16 bytes = 32x compression)
let codes = pq.encode(&vector)?;

Async API

use vecstore::{AsyncVecStore, Query};

#[tokio::main]
let store = AsyncVecStore::open("vectors.db").await?;

let query = Query::new(vec![0.1, 0.2, 0.3]).with_limit(10);
let results = store.query(query).await?;

Feature Flags

• async - Enable async API with Tokio support
• python - Enable Python bindings via PyO3
• embeddings - Enable built-in text embeddings via ONNX Runtime
• wasm - Enable WebAssembly support for browsers

Performance

vecstore is designed for high performance:

• Search: < 1ms for 100K vectors (on modern hardware)
• Insertion: ~1000 vectors/sec with HNSW indexing
• Memory: 8-32x reduction with Product Quantization
• Throughput: Parallel operations via Rayon

See BENCHMARKS.md for detailed performance analysis.

Architecture

vecstore uses:

• HNSW for approximate nearest neighbor search
• Product Quantization for memory-efficient vector compression
• BM25 for keyword-based text search
• Bincode for efficient binary serialization
• Rayon for parallel processing

Use Cases

• RAG applications (Retrieval-Augmented Generation)
• Semantic search over documents/images/code
• Recommendation systems
• Duplicate detection
• Clustering and classification

Re-exports

pub use collection::Collection;

pub use collection::CollectionConfig;

pub use collection::VecDatabase;

pub use error::Result;

pub use error::VecStoreError;

pub use graph_viz::GraphEdge;

pub use graph_viz::GraphNode;

pub use graph_viz::GraphStatistics;

pub use graph_viz::HnswVisualizer;

pub use namespace::Namespace;

pub use namespace::NamespaceId;

pub use namespace::NamespaceQuotas;

pub use namespace::NamespaceStatus;

pub use namespace::ResourceUsage;

pub use namespace_manager::AggregateStats;

pub use namespace_manager::NamespaceManager;

pub use namespace_manager::NamespaceStats;

pub use schema::FieldSchema;

pub use schema::FieldType;

pub use schema::Schema;

pub use schema::ValidationError;

pub use store::make_record;

pub use store::parse_filter;

pub use store::BatchError;

pub use store::BatchOperation;

pub use store::BatchResult;

pub use store::CompactionConfig;

pub use store::CompactionResult;

pub use store::Config;

pub use store::Distance;

pub use store::ExplainedNeighbor;

pub use store::FilterExpr;

pub use store::FilterOp;

pub use store::FilterParseError;

pub use store::HNSWSearchParams;

pub use store::HybridQuery;

pub use store::Metadata;

pub use store::Neighbor;

pub use store::PQConfig;

pub use store::PQVectorStore;

pub use store::PrefetchQuery;

pub use store::ProductQuantizer;

pub use store::Query;

pub use store::QueryEstimate;

pub use store::QueryExplanation;

pub use store::QueryPlan;

pub use store::QueryStage;

pub use store::QueryStep;

pub use store::Record;

pub use store::VecStore;

pub use store::VecStoreBuilder;

pub use text_splitter::RecursiveCharacterTextSplitter;

pub use text_splitter::TextChunk;

pub use text_splitter::TextSplitter;

pub use text_splitter::TokenTextSplitter;

pub use fuzzy::damerau_levenshtein_distance;

pub use fuzzy::levenshtein_distance;

pub use fuzzy::similarity_score;

pub use fuzzy::suggest_corrections;

pub use fuzzy::BKTree;

pub use fuzzy::FuzzyMatcher;

pub use autotuning::AutoTuner;

pub use autotuning::HnswParams;

pub use autotuning::PerformanceConstraints;

pub use autotuning::TuningGoal;

pub use compression::decode_rle;

pub use compression::decode_varint;

pub use compression::encode_rle;

pub use compression::encode_varint;

pub use compression::CompressedNeighborList;

pub use compression::CompressionConfig;

pub use compression::CompressionLevel;

pub use compression::CompressionMethod;

pub use compression::CompressionStats;

pub use realtime::BufferEntry;

pub use realtime::CompactionStats;

pub use realtime::RealtimeConfig;

pub use realtime::RealtimeIndex;

pub use realtime::RealtimeMetrics;

pub use realtime::Snapshot;

pub use realtime::UpdateStrategy;

pub use realtime::WorkerConfig;

pub use realtime::WriteBuffer;

pub use gpu::GpuBackend;

pub use gpu::GpuBenchmark;

pub use gpu::GpuConfig;

pub use gpu::GpuDeviceInfo;

pub use gpu::GpuExecutor;

pub use gpu::GpuOps;

pub use distributed::ConsistencyLevel;

pub use distributed::ConsistentHashRing;

pub use distributed::DistributedConfig;

pub use distributed::DistributedStats;

pub use distributed::DistributedStore;

pub use distributed::NodeInfo;

pub use distributed::NodeStatus;

pub use distributed::ReplicationStrategy;

pub use distributed::ShardInfo;

pub use distributed::ShardingStrategy;

pub use quantization::BinaryQuantizer;

pub use quantization::QuantizationBenchmark;

pub use quantization::ScalarQuantizer4;

pub use quantization::ScalarQuantizer8;

pub use ivf_pq::IVFPQConfig;

pub use ivf_pq::IVFPQIndex;

pub use ivf_pq::IVFPQStats;

pub use lsh::LSHConfig;

pub use lsh::LSHIndex;

pub use lsh::LSHStats;

pub use scann::ScaNNConfig;

pub use scann::ScaNNIndex;

pub use scann::ScaNNStats;

pub use timeseries::DecayFunction;

pub use timeseries::TemporalGroup;

pub use timeseries::TimeQuery;

pub use timeseries::TimeSeriesEntry;

pub use timeseries::TimeSeriesIndex;

pub use timeseries::TimeSeriesResult;

pub use timeseries::TimeSeriesStats;

pub use timeseries::WindowResult;

pub use multimodal::Modality;

pub use multimodal::MultiModalEntry;

pub use multimodal::MultiModalFusion;

pub use multimodal::MultiModalIndex;

pub use multimodal::MultiModalQuery;

pub use multimodal::MultiModalResult;

pub use multimodal::MultiModalStats;

pub use migration::MigrationRecord;

pub use migration::MigrationStats;

pub use migration::Migrator;

pub use migration::SourceDatabase;

pub use graph_rag::Entity;

pub use graph_rag::GraphQuery;

pub use graph_rag::GraphRAG;

pub use graph_rag::GraphResult;

pub use graph_rag::GraphStats;

pub use graph_rag::Relation;

pub use protocol::Protocol;

pub use protocol::ProtocolAdapter;

pub use protocol::UniversalRequest;

pub use protocol::UniversalResponse;

pub use protocol::VectorData;

pub use langchain::Document;

pub use langchain::LangChainVectorStore;

pub use langchain::LlamaIndexVectorStore;

pub use langchain::Node;

pub use langchain::RetrieverConfig;

pub use langchain::ScoredDocument;

pub use langchain::VectorStoreRetriever;

pub use benchmark::BenchmarkConfig;

pub use benchmark::BenchmarkResults;

pub use benchmark::Benchmarker;

pub use benchmark::ConcurrentResults;

pub use benchmark::FilterResults;

pub use benchmark::IndexingResults;

pub use benchmark::InsertResults;

pub use benchmark::LatencyStats;

pub use benchmark::MemoryResults;

pub use benchmark::QuantizationResults;

pub use benchmark::QueryResults;

pub use health::print_health_report;

pub use health::Alert;

pub use health::AlertCategory;

pub use health::AlertSeverity;

pub use health::DatabaseHealth;

pub use health::HealthCheckConfig;

pub use health::HealthChecker;

pub use health::HealthReport;

pub use health::HealthStatus;

pub use health::IndexHealth;

pub use health::PerformanceHealth;

pub use health::ResourceHealth;

pub use metadata_index::BTreeIndex;

pub use metadata_index::HashIndex;

pub use metadata_index::IndexConfig;

pub use metadata_index::IndexStats;

pub use metadata_index::IndexType;

pub use metadata_index::IndexedValue;

pub use metadata_index::InvertedIndex;

pub use metadata_index::MetadataIndex;

pub use metadata_index::MetadataIndexManager;

pub use clustering::ClusteringConfig;

pub use clustering::ClusteringResult;

pub use clustering::DBSCANClustering;

pub use clustering::DBSCANConfig;

pub use clustering::HierarchicalClustering;

pub use clustering::HierarchicalConfig;

pub use clustering::KMeansClustering;

pub use clustering::LinkageMethod;

pub use bulk_migration::BulkMigrationStats;

pub use bulk_migration::ChromaDBMigration;

pub use bulk_migration::FormatConverter;

pub use bulk_migration::MigrationConfig;

pub use bulk_migration::PineconeMigration;

pub use bulk_migration::QdrantMigration;

pub use partitioning::PartitionConfig;

pub use partitioning::PartitionInfo;

pub use partitioning::PartitionStats;

pub use partitioning::PartitionedStore;

pub use anomaly::AnomalyDetector;

pub use anomaly::AnomalyEnsemble;

pub use anomaly::AnomalyResult;

pub use anomaly::IsolationForest;

pub use anomaly::LocalOutlierFactor;

pub use anomaly::ZScoreDetector;

pub use dim_reduction::ReductionStats;

pub use dim_reduction::PCA;

pub use recommender::CollaborativeRecommender;

pub use recommender::ContentBasedRecommender;

pub use recommender::HybridRecommender;

pub use recommender::Recommendation;

pub use recommender::UserPreference;

pub use versioning::Snapshot as VersionSnapshot;

pub use versioning::Version;

pub use versioning::VersionDiff;

pub use versioning::VersionHistory;

pub use versioning::VersionedStore;

pub use versioning::VersioningStats;

pub use query_optimizer::CostBreakdown;

pub use query_optimizer::ExecutionPlan;

pub use query_optimizer::HintCategory;

pub use query_optimizer::Impact;

pub use query_optimizer::OptimizationHint;

pub use query_optimizer::QueryAnalysis;

pub use query_optimizer::QueryComparison;

pub use query_optimizer::QueryComplexity;

pub use query_optimizer::QueryOptimizer;

pub use query_optimizer::StoreOptimizationSummary;

pub use deduplication::BatchDeduplicator;

pub use deduplication::DeduplicationConfig;

pub use deduplication::DeduplicationStats;

pub use deduplication::DeduplicationStrategy;

pub use deduplication::Deduplicator;

pub use deduplication::DuplicateGroup;

pub use validation::BatchStatistics;

pub use validation::QualityMetrics;

pub use validation::ValidationConfig;

pub use validation::ValidationError as VectorValidationError;

pub use validation::ValidationResult;

pub use validation::ValidationStrictness;

pub use validation::ValidationWarning;

pub use validation::VectorValidator;

pub use analytics::AnalyticsConfig;

pub use analytics::AnalyticsReport;

pub use analytics::ClusterTendency;

pub use analytics::DimensionStats;

pub use analytics::DistributionStats;

pub use analytics::OutlierAnalysis;

pub use analytics::SimilarityStats;

pub use analytics::VectorAnalytics;

pub use monitoring::Alert as MonitorAlert;

pub use monitoring::AlertCategory as MonitorAlertCategory;

pub use monitoring::AlertCondition;

pub use monitoring::AlertPresets;

pub use monitoring::AlertRule;

pub use monitoring::AlertSeverity as MonitorAlertSeverity;

pub use monitoring::MetricHistory;

pub use monitoring::MetricPoint;

pub use monitoring::MetricType;

pub use monitoring::Monitor;

pub use monitoring::MonitoringConfig;

pub use monitoring::MonitoringReport;

pub use monitoring::MonitoringStats;

pub use rate_limit::MultiTierRateLimiter;

pub use rate_limit::RateLimitAlgorithm;

pub use rate_limit::RateLimitConfig;

pub use rate_limit::RateLimitResult;

pub use rate_limit::RateLimitScope;

pub use rate_limit::RateLimiter;

pub use audit::AuditBackend;

pub use audit::AuditConfig;

pub use audit::AuditEntry;

pub use audit::AuditEventType;

pub use audit::AuditLogger;

pub use audit::AuditMetadata;

pub use audit::AuditOutcome;

pub use audit::AuditSeverity;

pub use audit::FileBackend;

pub use audit::MemoryBackend;

pub use audit::StdoutBackend;

pub use access_control::AccessContext;

pub use access_control::AccessControl;

pub use access_control::Condition;

pub use access_control::Effect;

pub use access_control::Operator;

pub use access_control::Permission;

pub use access_control::Policy;

pub use access_control::Resource;

pub use access_control::Role;

pub use access_control::User;

pub use splade::SparseIndex;

pub use splade::SparseIndexStats;

pub use splade::SparseVector;

pub use splade::SpladeConfig;

pub use splade::SpladeEncoder;

pub use multi_vector::AggregationMethod;

pub use multi_vector::MultiVectorDoc;

pub use multi_vector::MultiVectorIndex;

pub use multi_vector::MultiVectorStats;

pub use geospatial::BoundingBox;

pub use geospatial::GeoDocument;

pub use geospatial::GeoIndex;

pub use geospatial::GeoIndexStats;

pub use geospatial::GeoPoint;

pub use geospatial::GeoSearchResult;

pub use advanced_filter::parse_advanced_filter;

pub use advanced_filter::AdvancedFilter;

pub use advanced_filter::FilterBuilder;

pub use profiler::ProfileStage;

pub use profiler::ProfileSummary;

pub use profiler::ProfilerConfig;

pub use profiler::QueryProfile;

pub use profiler::QueryProfiler;

pub use vectors::bm25_score;

pub use vectors::bm25_score_simple;

pub use vectors::hybrid_search_score;

pub use vectors::normalize_scores;

pub use vectors::normalize_scores_zscore;

pub use vectors::BM25Config;

pub use vectors::BM25Stats;

pub use vectors::FusionStrategy;

pub use vectors::HybridQuery as HybridQueryV2;

pub use vectors::HybridSearchConfig;

pub use vectors::KMeans;

pub use vectors::Vector;

pub use vectors::VectorOps;

pub use simd::cosine_similarity_simd;

pub use simd::dot_product_simd;

pub use simd::euclidean_distance_simd;

pub use simd::hamming_distance_simd;

pub use simd::jaccard_distance_simd;

pub use simd::jaccard_similarity_simd;

pub use simd::magnitude_simd;

pub use simd::manhattan_distance_simd;

Modules

access_control

Access Control and RBAC (Role-Based Access Control)

advanced_filter

Advanced Metadata Filtering

analytics

Vector analytics and statistical analysis

anomaly

Anomaly detection for vector outliers

audit

Audit Logging

autotuning

Auto-tuning for HNSW parameters

benchmark

Comprehensive benchmarking suite for VecStore

bulk_migration

Bulk migration tools for importing from other vector databases

cache

Query result caching with LRU eviction

clustering

Vector clustering algorithms

collection

Collection Abstraction for VecStore

compression

Index Compression

deduplication

Vector deduplication and near-duplicate detection

dim_reduction

Dimensionality reduction for vectors

distributed

Distributed Multi-Node Indexing

error

Error types for vecstore

fuzzy

Fuzzy search with typo tolerance

geospatial

Geospatial Indexing and Queries

gpu

GPU Acceleration for Vector Operations

graph_rag

Graph RAG - Knowledge Graph + Vector Search Integration

graph_viz

HNSW graph visualization and export

health

Health checks and diagnostics for VecStore

import_export

Data import/export in standard formats

ivf_pq

IVF-PQ Index Implementation

langchain

LangChain and LlamaIndex integration layer

lsh

Locality-Sensitive Hashing (LSH) Index

metadata_index

Metadata indexing for fast filtered queries

metrics

Observability and metrics for production monitoring

migration

Migration Tools for Vector Database Import/Export

mmap

Memory-mapped storage for disk-based HNSW index

monitoring

Vector Database Monitoring and Alerting

multi_vector

Multi-Vector Document Storage (ColBERT-style)

multimodal

Multi-Modal Vector Search

namespace

Multi-tenant namespace support with quotas and resource limits

namespace_manager

Namespace manager for multi-tenant vector database

partitioning

Vector partitioning for data isolation and multi-tenancy

profiler

Query Profiler with Flame Graph Support

protocol

Standard Vector Database Protocol

quantization

Scalar Quantization (SQ) and Binary Quantization (BQ) for extreme compression.

query_analyzer

Query performance analysis and explain plans

query_optimizer

Query cost estimation and optimization

rag_utils

RAG Utilities Module

rate_limit

Rate Limiting and Throttling

realtime

Real-time Index Updates

recommender

Recommender system for vector-based recommendations

reranking

Reranking Module

scann

ScaNN - Scalable Nearest Neighbors

schema

Schema validation for metadata

semantic_cache

Semantic caching with fuzzy key matching

simd

SIMD-accelerated distance calculations

splade

SPLADE: Sparse Lexical AnD Expansion Model

store

stream

Streaming query APIs for memory-efficient result iteration

telemetry

Distributed tracing and observability for VecStore

text_splitter

Text Splitting for RAG Applications

timeseries

Time-Series Vector Search

tokenizer

Pluggable tokenizer system for text processing

validation

Vector validation and quality checks

vectors

Vector representations and operations

versioning

Vector versioning and rollback support

wal

Write-Ahead Logging (WAL) for crash recovery

Functions

init_tracing

Initialize tracing subscriber for logging