Struct InvertedIndex 

pub struct InvertedIndex { /* private fields */ }

Inverted index for lexical search

Implementations

impl InvertedIndex

pub fn new(config: BM25Config) -> Self

Create a new inverted index

pub fn with_positions(self) -> Self

Enable position storage (for phrase queries)

pub fn add_document(&self, text: &str) -> DocId

Index a document

Returns the assigned document ID.

pub fn add_document_with_id(&self, doc_id: DocId, text: &str)

Index a document with specific ID

pub fn clear(&self)

Remove every document, returning the index to its freshly-created state while preserving the BM25 configuration and position-storage setting.

pub fn rebuild_from_documents<'a, I>(&self, documents: I)

where I: IntoIterator<Item = (DocId, &'a str)>,

Rebuild the entire index from an authoritative (doc_id, text) source.

Durability contract (Task 7)

This index is an in-memory derived structure: it holds no WAL and is not itself crash-durable. The committed document store is the source of truth, and lexical search agrees with it only up to the last rebuild. The supported recovery model is therefore:

1. Documents commit through the durable storage path (WAL/MVCC).
2. On restart, the lexical index is reconstructed from the committed document store via this method — an O(corpus) bounded, deterministic pass (it clears first, so the result is a pure function of the input, independent of any prior in-memory state).
3. Only then is lexical search served.

Because scores are order-independent (IDF is derived from (df, N) and avgdl from running totals, never cached), a rebuilt index is byte-for-byte equivalent in ranking to the index that produced the documents, so a query returns the same result set across a crash/restart boundary relative to one committed snapshot.

pub fn add_document_tokens(&self, tokens: &[String]) -> DocId

Index a document from tokens

pub fn add_document_tokens_with_id(&self, doc_id: DocId, tokens: &[String])

Index a document from tokens with specific ID

pub fn remove_document(&self, doc_id: DocId) -> bool

Remove a document from the index

pub fn search(&self, query: &str, limit: usize) -> Vec<(DocId, f32)>

Search the index

Returns document IDs with scores, sorted by score descending.

pub fn search_tokens( &self, query_tokens: &[String], limit: usize, ) -> Vec<(DocId, f32)>

Search with pre-tokenized query

pub fn get_posting_list(&self, term: &str) -> Option<PostingList>

Get posting list for a term

pub fn num_documents(&self) -> usize

Get document count

pub fn vocab_size(&self) -> usize

Get vocabulary size

pub fn get_document_info(&self, doc_id: DocId) -> Option<DocumentInfo>

Get document info

pub fn has_document(&self, doc_id: DocId) -> bool

Check if a document exists