Module collection 

A collection of vectors under keys, which is what both doors reach.

Partitions is the index and it knows about ids, dimensions and codes. It does not know that a client calls a vector doc:1, that a collection has a metric, or where the full precision vectors live, because none of those are the index’s business and putting them there would make one of the two callers wrong. This is the piece above it that answers all three, so db.vectors() and VADD off a socket are two doors into one store (Y23) rather than two stores that agree for now.

use yo_vector::Collection;
use yo_shape::Metric;

let mut c = Collection::new(3, Metric::L2)?;
c.put(b"a", &[1.0, 0.0, 0.0])?;
c.put(b"b", &[0.0, 1.0, 0.0])?;

let hits = c.search(&[0.9, 0.1, 0.0], 1, None)?;
assert_eq!(hits[0].key, b"a");

The metric decides what is stored

Metric::L2 stores the vector it was given. Metric::Cosine stores the unit vector, because the index measures distance and on unit vectors the nearest by distance is the nearest by angle, so cosine costs one normalisation on the way in rather than a second index. That is also what comes back out of Collection::get, and it is the same answer Redis gives for a cosine vector set.

Metric::Ip and Metric::Hamming are refused rather than approximated. Inner product is not a distance, so ordering by it is not ordering by nearness and the partitions would be built around the wrong question. Hamming wants binary vectors that a collection of floats does not hold.

Where the vectors live

Beside the index, one flat run of floats with a slot per id, which the last step of every search reads and which the index itself never holds. An id is never anything but the slot it names, so the rerank is an offset rather than a lookup, and a slot comes back for reuse when its key is removed. 06 puts them in the record log at kind 3 and yo_format::vector is that record already written down. Until the file lands the run is in memory, which is where every other collection in this build is too, and nothing on this page changes when it moves.

Structs

Collection

Vectors under keys: the index, the vectors it reranks against, and the table that turns one into the other.

Match

One answer from a search.

Constants

MAX_DIM

The largest dimension a collection can hold.

Functions

check_metric

Whether metric is one this build can measure.

width

dim as the shape grammar writes it, if it is a dimension a collection can be opened with.