# Weavatrix Search Vector
[](https://crates.io/crates/weavatrix-search-vector)
[](https://docs.rs/weavatrix-search-vector)
[](https://github.com/sergii-ziborov/weavatrix-search-vector/actions/workflows/ci.yml)
[](https://github.com/sergii-ziborov/weavatrix-search-vector/blob/main/LICENSE)
`weavatrix-search-vector` is a first-party vector-candidate engine for
Weavatrix and other Rust applications. It provides deterministic hybrid HNSW
plus multi-probe `SimHash`, exact and scalar-int8 search, versioned persistence,
read-only memory mapping, mutable overlays, metadata filters, embedding
adapters, KNN graph construction, and bounded shard coordination.
The public API is safe Rust, requires Rust 1.88, and has no runtime
dependencies, native libraries, helper processes, or external vector engines.
Two private audited modules isolate runtime-detected `std::arch` kernels and
read-only OS mapping on Windows, Linux, and macOS. All other code remains safe
Rust.
## Installation
```text
cargo add weavatrix-search-vector
```
Or add the crate directly to `Cargo.toml`:
```toml
[dependencies]
weavatrix-search-vector = "0.2"
```
## Boundary
Vector Search owns:
- vector and query validation;
- deterministic HNSW construction;
- uncertainty-driven `SimHash` candidate recovery;
- approximate and exact top-K candidate search;
- stable equal-distance ordering by caller-provided `u64` key;
- bounded batch workers;
- versioned snapshots with payload checksums;
- owned and zero-copy read-only snapshot loading;
- incremental upsert/delete overlays with deterministic compaction;
- composable metadata predicates with exact filtered fallback;
- scalar-int8 candidates with optional exact f32 reranking;
- model-neutral embedding provider integration;
- directed KNN candidate graph construction;
- transport-neutral bounded shard fan-out and stable merging;
- recall and retained-allocation evidence.
It does not know about the Weavatrix domain graph, semantic thresholds,
embedding models, provenance, mutual/union policies, text search, or repository
discovery. Semantic consumers remain responsible for exact rescoring and
relationship policy.
## Example
```rust
use weavatrix_search_vector::{IndexConfig, VectorIndex};
let first = [1.0, 0.0, 0.0];
let second = [0.9, 0.1, 0.0];
let third = [0.0, 1.0, 0.0];
let vectors = [(10, first.as_slice()), (20, second.as_slice()), (30, third.as_slice())];
let index = VectorIndex::build(IndexConfig::new(3), &vectors)?;
let hits = index.search(&[1.0, 0.0, 0.0], 2)?;
assert_eq!(hits[0].key, 10);
assert_eq!(hits[1].key, 20);
# Ok::<(), weavatrix_search_vector::SearchError>(())
```
`VectorIndex::search_batch` preserves query order and reuses one visited set
and heap set per bounded standard-library worker. `VectorIndex::search_exact`
and `ExactIndex` provide deterministic ground truth for recall tests.
## Persistence and live updates
```rust,no_run
use weavatrix_search_vector::{
IndexConfig, MappedVectorIndex, Metadata, MutableVectorIndex, VectorIndex,
VectorRecord,
};
let vector = [1.0, 0.0, 0.0];
let index = VectorIndex::build(IndexConfig::new(3), &[(10, vector.as_slice())])?;
index.save("vectors.wvx")?;
let mapped = MappedVectorIndex::open("vectors.wvx")?;
assert_eq!(mapped.search(&vector, 1)?[0].key, 10);
let mutable = MutableVectorIndex::build(
IndexConfig::new(3),
&[VectorRecord::new(10, vector.to_vec())],
)?;
mutable.upsert(20, &[0.9, 0.1, 0.0], Metadata::new())?;
mutable.delete(10);
mutable.compact()?;
# Ok::<(), weavatrix_search_vector::SearchError>(())
```
`VectorIndex::save` writes normalized vectors, routing buckets, and all HNSW
replicas through a flushed temporary sibling. `MappedVectorIndex` validates
the version, ranges, graph references, finite values, and checksum before
serving queries directly from mapped vectors and graph arrays. Structure-only
validation is available for trusted immutable artifacts.
`MutableVectorIndex` uses an exact delta and tombstones over an immutable base.
Readers never observe a partial compaction; three conflicting optimistic
rebuilds return a typed `MutationConflict`.
## Extended search primitives
- `MetadataIndex` and `MetadataFilter` compose equality, existence, numeric
ranges, text prefixes, Boolean operations, and exact fallback.
- `ScalarQuantizedIndex` stores one signed byte per component and can rerank a
wider candidate set against `VectorIndex`.
- `EmbeddingProvider` and `EmbeddingIndex` batch external local or hosted
encoders without selecting a model or adding a runtime dependency.
- `KnnGraph` builds deterministic directed candidate edges in compressed-row
form; semantic thresholds and graph meaning stay with the consumer.
- `SearchShard` and `DistributedIndex` merge heterogeneous local, mapped,
quantized, mutable, or remote-backed shards under one worker budget.
## Algorithm
Each HNSW replica uses:
- key-sorted normalized vector storage;
- seed/key-derived levels and insertion permutation;
- deterministic parallel bulk-construction waves;
- greedy upper-layer descent;
- bounded best-first layer search;
- diversified outgoing links and retained reverse links;
- doubled layer-zero outgoing degree;
- a compact 14-bit `SimHash` table with five uncertainty-driven probes;
- runtime-dispatched first-party cosine kernels;
- exact cosine distances for every returned hit.
Independent replicas and the construction work inside each replica share the
configured build-worker budget. The built index is immutable, `Send`, and
`Sync`. Memory is proportional to vector storage and retained graph links,
never to the square of the vector count.
## Reference benchmark
The reference gate is 10,000 vectors x 384 dimensions, cosine distance,
top-8, one warm-up and three release runs:
- build plus all 10,000 approximate queries at most 3 seconds on the reference
Windows host;
- recall@8 at least 99.9% against the exact oracle;
- retained allocation estimate below 256 MiB;
- identical API behavior on Windows, Linux, and macOS;
- Rust 1.88, rustfmt, Clippy and rustdoc with warnings denied.
The 2026-07-27 Windows run passes the local performance, recall, and allocation
gates:
| Median build, 3 runs | 197.007 ms |
| Median all-query search, 3 runs | 81.500 ms |
| Median build + search, 3 runs | 278.508 ms |
| Full-oracle recall@8, all 10,000 queries | 99.9888% |
| Estimated retained index allocation | 18.037 MiB |
Machine: Intel Core Ultra 7 255U, 12 cores / 14 logical processors, Windows 11
Enterprise 10.0.26200, `rustc 1.97.1` GNU. The corpus is deterministic,
synthetic, and clustered. The allocation figure is calculated from retained
vector/link capacities; it is not process RSS. See
[`docs/benchmark-2026-07-27.md`](https://github.com/sergii-ziborov/weavatrix-search-vector/blob/main/docs/benchmark-2026-07-27.md)
for commands,
raw results, and limitations.
```text
cargo bench --bench vector_search -- run
```
Environment variables `WV_VECTOR_COUNT`, `WV_VECTOR_DIMENSIONS`,
`WV_VECTOR_TOP_K`, `WV_VECTOR_RUNS`, `WV_VECTOR_EXACT_QUERIES`,
`WV_VECTOR_CONNECTIVITY`, `WV_VECTOR_EXPANSION_BUILD`,
`WV_VECTOR_EXPANSION_QUERY`, and `WV_VECTOR_REPLICAS` control the reproducible
corpus and policy. Set `WV_VECTOR_EXACT_QUERIES=10000` for the complete
reference recall calculation.
These figures establish this crate's disclosed workload, not a general speed
claim against another vector engine. Cross-engine benchmarks require identical
vectors, query sets, thread budgets, recall, and process-memory measurement.
## Competitors
The five-run quality-gated comparison against `hnsw_rs 0.3.4` and
`usearch 2.26.0` found:
| Weavatrix | **232 ms** | **91 ms** | **321 ms** | 99.9888% |
| `hnsw_rs` | 892 ms | 927 ms | 1,819 ms | 99.9687% |
| `usearch` | 1,971 ms | 157 ms | 2,128 ms | 99.9925% |
Weavatrix is fastest in build, query-only, and build-plus-query on the
reference corpus while staying above the 99.9% recall gate. At 50,000 x 384,
Weavatrix retained 99.95% sampled recall with a 0.956-second all-query pass;
the tested `usearch` policies were slower and did not retain a 99.9%
three-run minimum.
See
[`docs/competitive-benchmark-2026-07-27.md`](https://github.com/sergii-ziborov/weavatrix-search-vector/blob/main/docs/competitive-benchmark-2026-07-27.md)
for policies, three-run minimum recall, full-oracle evidence, memory caveats,
functional gaps, and reproduction commands.
## 0.2 feature evidence
On the same 10,000 x 384 corpus, the median of three separately launched
release runs opened the fully checksummed mapped snapshot in 50.096 ms,
executed 1,000 mapped HNSW queries in 42.577 ms, applied 100 in-memory upserts
in 0.165 ms, and compacted them in 241.023 ms. The 16.850 MiB snapshot and
3.777 MiB scalar-int8 retained estimate are different representations and
should not be compared as equal-recall algorithms.
See
[`docs/benchmark-0.2.0-2026-07-27.md`](https://github.com/sergii-ziborov/weavatrix-search-vector/blob/main/docs/benchmark-0.2.0-2026-07-27.md)
for raw runs, operation definitions, the full-recall core gate, and
limitations.
## Deliberate boundaries
The crate does not bundle embedding models, tokenizers, GPU runtimes, network
transports, distributed consensus, semantic thresholds, provenance policy, or
the Weavatrix domain graph. It supplies the vector, storage, mutation,
filtering, candidate-graph, and shard-coordination primitives those layers use.