pub struct SparseInvertedIndex { /* private fields */ }Expand description
An in-memory inverted index over sparse vectors (ADR-0045).
Maps each sparse dimension to the documents that have a nonzero weight there,
so search accumulates a dot-product score over
only the query’s nonzero dimensions. Built and maintained by quiver-embed;
never persisted.
Implementations§
Source§impl SparseInvertedIndex
impl SparseInvertedIndex
Sourcepub fn upsert(&mut self, ext_id: &str, sv: &SparseVector)
pub fn upsert(&mut self, ext_id: &str, sv: &SparseVector)
Insert or replace ext_id’s sparse vector. Re-upserting an existing id
first removes its prior postings, so a dimension it no longer carries does
not linger and a changed weight is not double-counted. Duplicate input
dimensions are de-duplicated (last weight wins).
Sourcepub fn remove(&mut self, ext_id: &str) -> bool
pub fn remove(&mut self, ext_id: &str) -> bool
Remove ext_id and free its slot. Returns whether it was present.
Sourcepub fn search(&self, query: &SparseVector) -> Vec<(String, f32)>
pub fn search(&self, query: &SparseVector) -> Vec<(String, f32)>
Score every document that shares a nonzero dimension with query by
sparse dot product, and return (ext_id, score) for those with a positive
score, sorted by score descending then id ascending (a deterministic, total
order). The caller re-checks any payload filter on the ranked ids and
truncates to its depth, so low-scored rows never load a payload. Duplicate
query dimensions are de-duplicated (last weight wins).
Sourcepub fn bm25_search(
&self,
query_terms: &[u32],
k1: f32,
b: f32,
) -> Vec<(String, f32)>
pub fn bm25_search( &self, query_terms: &[u32], k1: f32, b: f32, ) -> Vec<(String, f32)>
Score documents against query_terms (term ids) with Okapi BM25
(ADR-0046), treating each document’s stored weights as term frequencies and
using the index’s own corpus statistics — document frequency (posting-list
length), document count, and average document length. k1/b are the usual
BM25 parameters (BM25_K1, BM25_B). Duplicate query terms count once.
Returns (ext_id, score) for documents with a positive score, sorted by
score descending then id ascending. Uses the Lucene-style smoothed IDF
ln(1 + (N − df + 0.5)/(df + 0.5)), which is always non-negative, so even a
term in most of the corpus contributes a small positive amount (no negative
scores to clamp).