pub struct TextSegment { /* private fields */ }Expand description
One shard’s inverted segment.
postings maps token → (key → tf) so a pruned list is PROBED per
accumulated candidate (O(candidates)) instead of walked
(O(postings)); docs keeps each row’s original text so an update
removes exactly its own tokens (re-tokenize the old text) instead
of scanning every posting list.
Implementations§
Source§impl TextSegment
impl TextSegment
Sourcepub fn apply(&mut self, key: &[u8], text: Option<&[u8]>)
pub fn apply(&mut self, key: &[u8], text: Option<&[u8]>)
(Re-)index one row’s text (None = row removed / excluded).
Sourcepub fn matches(&self, query: &[u8], limit: usize) -> Vec<TextMatch>
pub fn matches(&self, query: &[u8], limit: usize) -> Vec<TextMatch>
BM25-ranked matches for query (tokenized with the same rules;
OR semantics), best limit hits, score-descending.
MaxScore pruning: query tokens process rarest-first; once the
running top-limit threshold exceeds the summed upper bounds
of the remaining (commoner) tokens, documents seen ONLY in
those lists can no longer enter — their lists are then probed
per accumulated doc instead of walked. Selection is a bounded
heap over borrowed keys (no per-candidate allocation).