warm-rust 0.6.5

Local-first search engine and agent memory: fast approximate nearest-neighbor and BM25 full-text search, encrypted stores, provable deletion, built on vector-symbolic architectures.
Documentation
# Capacity limits

## How many items fit in a bundle?

A bundle superposes items into one fixed-size vector, so it has a capacity: past some number of
items it can no longer tell members from non-members. This page reports what was measured and
where the measurement stops. It does not extrapolate.

## What was measured

[`benchmark_scaling_results.json`](https://github.com/writerslogic/warm/blob/main/benchmarks/results/benchmark_scaling_results.json)
(generated by `warm-scaling`, 2026-06-19) stores `N` random items in one union (Bloom) bundle, then
probes it with 50 members and 50 non-members per `N`. A **positive gap** means members still score
above non-members on average. **d' >= 2** means the separation between the two score
distributions is at least two noise standard deviations. The table gives the last `N` in the
sweep that meets each criterion, with the measured false-positive rate there.

| Dimension | Density | Active bits | Last N, positive gap | False positives there | Last N, d' >= 2 | False positives there | Largest N tested |
|-----------|---------|-------------|----------------------|-----------------------|-----------------|-----------------------|------------------|
| 16,384 | 1/256 | 64 | 500 | 4% | 1,000 | 32% | 2,500 |
| 65,536 | 1/256 | 256 | 1,000 | 4% | 1,000 | 4% | 3,000 |
| 65,536 | 1/1024 | 64 | 2,800 | 14% | 4,000 | 26% | 10,000 |
| 131,072 | 1/256 | 512 | 1,000 | 6% | 1,500 | 26% | 3,000 |
| 131,072 | 1/1024 | 128 | 2,800 | 8% | 4,000 | 30% | 14,000 |
| 262,144 | 1/1024 | 256 | 4,000 | 12% | 4,000 | 12% | 14,000 |
| 262,144 | 1/4096 | 64 | 11,200 | 12% | 16,000 | 18% | 64,000 |
| 524,288 | 1/1024 | 512 | 4,000 | 2% | 4,000 | 2% | 14,000 |
| 524,288 | 1/4096 | 128 | 12,800 | 16% | 16,000 | 30% | 56,000 |

## How to read it

- **A "last N" is not a safe capacity.** At D = 16,384 the false-positive rate is already 4% at
  N = 500 and 32% at N = 1,000. A deployment that needs a low false-positive rate has to stay
  well below these points, or verify candidates against the exact store.
- **Earlier numbers were wrong.** Previous versions of the README reported 2,478, 9,800 and 58,432
  as a "hard wall". Those are the points where the bundle is saturated and the false-positive rate
  is 100%. They are not usable capacities and are no longer quoted.
- **Sparser is roomier, up to a point.** The 1/4096 density rows hold far more items than 1/256 at
  the same dimension, with fewer active bits per item (64 at D = 262,144, 128 at D = 524,288).
- **Probes are small.** 50 members and 50 non-members per point means a measured rate of 4% is
  two false positives. Treat single-digit percentages as coarse.
- **The sweep grid differs per row.** Each row's range is in the last column; nothing is
  claimed beyond it.

## What this does not establish

- The test items are random vectors. Real text encodes to clustered, correlated vectors; the effect on
  capacity has not been measured here.
- It is a membership test on a union bundle. Per-fact retrieval of a value bound to a key from a
  superposition has its own, lower floor (the linked note reports roughly M/D of 0.1 to 0.3 for naive unbinding); see the
  [superposition floor note]../SUPERPOSITION-FLOOR.md.
- It does not describe the document store. Documents are stored one chunk at a time, not
  superposed, so their limits are memory and the resource limits in
  [Production readiness]../production-readiness.md.

## Reproduce

```sh
cargo run --release --features experimental --bin warm-scaling -- --dim 16384 --density 256 --json
```