spacedb-vector 0.7.3

SpaceDB Layer 4 — the private-RAG substrate: an on-node vector index where a query embedding goes in and only top-k results come out (ids + scores), so the corpus never leaves the home. Retrieval is gated by an mID capability. Phase 1 SpaceDB Mission, M6.
Documentation

spacedb-vector

crates.io docs.rs License: MIT OR Apache-2.0 Remade With Rust By Mata Network

SpaceDB Layer 4 — the private-RAG substrate.

A native, on-node vector index co-located with the data: a query embedding goes in, and only the top-k results come out. The corpus never leaves the home.

Part of SpaceDB. Dual-licensed MIT OR Apache-2.0.

[dependencies]

spacedb-vector = "0.7"

The model

use spacedb_vector::{retrieve, Metric, VectorIndex};
use spacedb_access::Decision;
let embedding_a: Vec<f32> = (0..384).map(|i| i as f32).collect();
let embedding_b: Vec<f32> = (0..384).map(|i| (384 - i) as f32).collect();
let query_embedding = embedding_a.clone();
let decision = Decision::Allow;   // from spacedb-access `authorize`

let mut index = VectorIndex::new(384, Metric::Cosine);
index.insert("note-1", embedding_a).unwrap();
index.insert("note-2", embedding_b).unwrap();

// An AI agent's retrieval is gated by an mID capability decision.
let hits = retrieve(&index, &query_embedding, 5, &decision).unwrap();
for m in hits {
    println!("{} {:.3}", m.id, m.score);   // ids + scores — nothing else
}

retrieve refuses with VectorError::Denied unless the spacedb-access Decision allows it. That is the concrete mechanism behind "inaccessible by default, accessible by mID-gated consent": an authorized agent gets semantic access to results, not bulk data.

Metrics: Cosine (magnitude-invariant, [-1, 1]), Dot, and Euclidean (negated, so higher is always better and k-selection is uniform).

What ships today

An exact flat k-NN index — insert / remove / search, dimension-checked on every operation. Sub-linear ANN (HNSW / IVF) is a future optimization behind the same surface: exactness first, so correctness is never traded for a recall knob nobody measured.

Open-core boundary

Depends only on spacedb-access, for the capability gate. No MATA crate.

Performance (0.7.2)

Measured on exact instruction counts (callgrind) with deterministic drivers whose outputs did not change; release build, 0.7.1 -> 0.7.2: ir_vector 33.1M -> 23.3M (-29.8%).

  • Dot product and L2 run on eight independent accumulators over fixed chunks, so they vectorize. An f32 score can differ from 0.7.1 in its last bits; top-k sets match serial scoring in the tests.
  • The id-to-position map hashes with FNV.

Performance (0.7.0)

Measured on exact instruction counts (callgrind) with this crate's deterministic driver, examples/ir_vector.rs; outputs unchanged by every change. Release build, 0.6.0 -> 0.7.0: 204.2M -> 33.1M instructions (-83.8%).

  • Insert finds an existing id through a position map (a bulk load was O(n^2)); remove keeps order.
  • Search ranks (score, position) pairs and selects the top k before sorting them; ids are copied only for the k results. Results, scores and tie order are unchanged (a NaN score falls back to the full sort).
  • Cosine norms are computed once per query and once per entry at insert.

Testing

The workspace defaults to wasm32; this crate is native. Test on your host triple:

cargo test -p spacedb-vector --target aarch64-apple-darwin   # or your host triple

Suite: vector.rs — metric correctness, top-k ordering, dimension mismatch, and the denied-without-capability path.

License

MIT OR Apache-2.0, at your option. See LICENSE-MIT and LICENSE-APACHE.