Struct FtsIndex 

pub struct FtsIndex<B>
where
    B: FtsBackend,
{ /* private fields */ }

Full-text search index generic over storage backend.

Indexing, search, and highlighting use the FtsBackend storage contract.

Writes accumulate in an in-memory Memtable, one state per index. Threshold-triggered flushing writes each index’s postings to an immutable segment of that index through the backend. Queries merge the active memtable with stored segments. The backend determines segment durability.

MemoryGovernor enforces per-engine memory budgets on large allocations (compaction, segment merge, query term collection).

Implementations

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn highlight( &self, text: &str, query: &str, prefix: &str, suffix: &str, ) -> String

Generate highlighted text with matched query terms wrapped in tags.

Returns the original text with each occurrence of a matched query term surrounded by prefix and suffix (e.g., <b> and </b>).

pub fn offsets(&self, text: &str, query: &str) -> Vec<MatchOffset>

Return byte offsets of matched query terms in the original text.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn set_collection_analyzer( &self, database_id: u64, tid: u64, collection: &str, analyzer_name: &str, ) -> Result<(), <B as FtsBackend>::Error>

Set the analyzer for a collection. Persists to backend metadata.

pub fn set_collection_language( &self, database_id: u64, tid: u64, collection: &str, lang_code: &str, ) -> Result<(), <B as FtsBackend>::Error>

Set the language for a collection. Persists to backend metadata.

pub fn set_collection_fuzzy( &self, database_id: u64, tid: u64, collection: &str, fuzzy: bool, ) -> Result<(), <B as FtsBackend>::Error>

Set whether searches over this collection fall back to fuzzy (Levenshtein) matching by default. Persists to backend metadata.

pub fn get_collection_fuzzy( &self, database_id: u64, tid: u64, collection: &str, ) -> Result<bool, <B as FtsBackend>::Error>

Whether this collection defaults to fuzzy matching. false when unset.

pub fn get_collection_analyzer( &self, database_id: u64, tid: u64, collection: &str, ) -> Result<Option<String>, <B as FtsBackend>::Error>

Get the configured analyzer name for a collection.

pub fn get_collection_language( &self, database_id: u64, tid: u64, collection: &str, ) -> Result<Option<String>, <B as FtsBackend>::Error>

Get the configured language for a collection.

pub fn analyze_for_collection( &self, database_id: u64, tid: u64, collection: &str, text: &str, ) -> Result<Vec<String>, <B as FtsBackend>::Error>

Analyze text using the collection’s configured analyzer.

Falls back to the standard English analyzer if no analyzer is configured.

pub fn tokenize_raw_for_collection( &self, database_id: u64, tid: u64, collection: &str, text: &str, ) -> Result<Vec<String>, <B as FtsBackend>::Error>

Tokenize text WITHOUT stemming for fuzzy matching.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn read_fieldnorm<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, doc_id: Surrogate, ) -> Result<Option<u32>, <B as FtsBackend>::Error>

Get the fieldnorm (SmallFloat-encoded doc length) for a doc.

Returns the decoded approximate u32 length, or None if not stored.

pub fn write_fieldnorm<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, doc_id: Surrogate, doc_length: u32, ) -> Result<(), <B as FtsBackend>::Error>

Write a fieldnorm byte for a surrogate. Grows the array if needed.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn index_stats<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, ) -> Result<(u32, f32), <B as FtsBackend>::Error>

Get total document count and average document length of one index.

Returns (total_docs, avg_doc_len). If the index is empty, returns (0, 1.0) to avoid division by zero.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn put_synonym_group( &self, database_id: u64, tid: u64, record: &SynonymGroupRecord, ) -> Result<(), <B as FtsBackend>::Error>

Persist a synonym group. Overwrites any existing group with the same name.

pub fn delete_synonym_group( &self, database_id: u64, tid: u64, name: &str, ) -> Result<bool, <B as FtsBackend>::Error>

Delete a synonym group. Returns true if it existed.

pub fn get_synonym_group( &self, database_id: u64, tid: u64, name: &str, ) -> Result<Option<SynonymGroupRecord>, <B as FtsBackend>::Error>

Read a single synonym group by name. Returns None if not found or tombstoned.

pub fn list_synonym_groups( &self, database_id: u64, tid: u64, ) -> Result<Vec<SynonymGroupRecord>, <B as FtsBackend>::Error>

List all synonym group records for a tenant.

pub fn build_synonym_map_for_tenant( &self, _database_id: u64, _tid: u64, all_groups: &[SynonymGroupRecord], ) -> SynonymMap

Build an in-memory SynonymMap from a slice of synonym group records.

Each term in every group maps to all other terms in that group (bidirectional OR-expansion). Terms are analyzed with the default analyzer so synonym keys match the stemmed tokens produced at query time by search_with_params.

pub fn expand_query_with_synonyms( &self, database_id: u64, tid: u64, tokens: Vec<String>, ) -> Result<Vec<String>, <B as FtsBackend>::Error>

Load all synonym groups for a tenant and build the expansion map.

Called at FTS query time inside search_with_params to expand query tokens before BM25 scoring.

pub fn expand_query_groups( &self, database_id: u64, tid: u64, tokens: Vec<String>, ) -> Result<Vec<Vec<String>>, <B as FtsBackend>::Error>

Expand each analyzed query token into its word group: the token first, then its distinct synonyms. One group per token, in token order. A document matches a word when it holds any term of the group.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn index_document<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, doc_id: Surrogate, text: &str, ) -> Result<(), FtsIndexError<<B as FtsBackend>::Error>>

Index a document’s text content into one index. The analyzer is the one bound to the index’s collection.

Returns Err(FtsIndexError::SurrogateOutOfRange) for Surrogate::ZERO or values exceeding MAX_INDEXABLE_SURROGATE. Surrogate::ZERO is the unassigned sentinel. Fieldnorm arrays use raw u32 surrogates as indexes. Values near u32::MAX cause multi-GiB allocations. Runtime bounds checks run before analysis and remain active in release builds.

pub fn index_analyzed_document<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, doc_id: Surrogate, tokens: &[String], ) -> Result<(), FtsIndexError<<B as FtsBackend>::Error>>

Index ordered tokens from this collection’s current analyzer. Callers preserve token order and hold analyzer config stable through indexing.

Returns Err(FtsIndexError::SurrogateOutOfRange) for Surrogate::ZERO or values exceeding MAX_INDEXABLE_SURROGATE.

pub fn remove_document<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, doc_id: Surrogate, ) -> Result<(), FtsIndexError<<B as FtsBackend>::Error>>

Remove a document from one index.

The memtable drops the document’s postings. Postings already flushed to a segment stay in that immutable segment, so the document enters the delete set of every segment the index holds: reads skip those postings and a merge drops them. The corpus stats lose the document’s length. A document the index does not hold changes nothing.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn flush_memtable<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, ) -> Result<(), FtsIndexError<<B as FtsBackend>::Error>>

Flush one index’s memtable postings to an immutable segment of that index in the backend. Every other index keeps its memtable state.

The postings leave the memtable only after the segment is written, so a failed flush loses nothing.

Calling this before serializing the index guarantees that all posting data written since the last spill threshold is captured in the backend’s segment storage rather than the in-memory memtable. Callers that checkpoint the index (e.g., NodeDB-Lite flush) must call this once per active index before persisting.

pub fn flush_all_memtables( &self, ) -> Result<(), FtsIndexError<<B as FtsBackend>::Error>>

Flush every index the memtable holds, each to its own segment.

pub fn purge_collection( &self, database_id: u64, tid: u64, collection: &str, ) -> Result<usize, <B as FtsBackend>::Error>

Purge every index of a collection. Returns count of removed entries.

pub fn purge_tenant( &self, database_id: u64, tid: u64, ) -> Result<usize, <B as FtsBackend>::Error>

Purge all entries for a (database_id, tenant) across every collection.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn new(backend: B, governor: Arc<MemoryGovernor>) -> FtsIndex<B>

Create a new FTS index with the given backend, default BM25 params, and a memory governor.

pub fn with_params( backend: B, params: Bm25Params, governor: Arc<MemoryGovernor>, ) -> FtsIndex<B>

Create a new FTS index with custom BM25 parameters and a memory governor.

pub fn with_memtable_config( backend: B, memtable: MemtableConfig, governor: Arc<MemoryGovernor>, ) -> FtsIndex<B>

Create a new FTS index whose memtable spills at memtable thresholds. Unreachable thresholds keep every posting in the memtable.

pub fn with_config( backend: B, params: Bm25Params, memtable: MemtableConfig, governor: Arc<MemoryGovernor>, ) -> FtsIndex<B>

Create a new FTS index with custom BM25 parameters whose memtable spills at memtable thresholds.

pub fn backend(&self) -> &B

Access the underlying backend.

pub fn backend_mut(&mut self) -> &mut B

Mutable access to the underlying backend.

pub fn memtable(&self) -> &Memtable

Access the active memtable (for LSM query merging).

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn search<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, params: FtsSearchParams<'_>, ) -> Result<Vec<TextSearchResult>, FtsIndexError<<B as FtsBackend>::Error>>

Search one index with explicit boolean mode, fuzzy, and optional prefilter.

Analyzer, fuzzy default, and synonyms come from the index’s collection. Postings and BM25 stats come from the index itself.

Supports NOT <term> and -<term> negation in the query string. Returns Err(FtsIndexError::InvalidQuery) for ill-formed queries such as NOT-only queries or unsupported parenthesised groups.

pub fn search_staged<'a>( &self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, params: FtsSearchParams<'_>, staged: Option<&StagedView>, ) -> Result<Vec<TextSearchResult>, FtsIndexError<<B as FtsBackend>::Error>>

Self::search inside an open transaction: indexed documents the transaction hides do not match, and its staged documents compete in the same ranking under the same query semantics. prefilter bounds staged documents too.

Results are ordered by score descending, then surrogate ascending.

impl<B> FtsIndex<B>

where B: FtsBackend,

pub fn doc_scorer<'a>( &'a self, database_id: u64, tid: u64, index: impl Into<IndexScope<'a>>, query: TextQuery<'_>, eligible: Option<&SurrogateBitmap>, staged: Option<StagedView>, ) -> Result<DocScorer<'a, B>, FtsIndexError<<B as FtsBackend>::Error>>

A scorer of query against index. eligible is the set of rows the reading query admits: the AND-mode fallback is decided over it, the same way a search with that prefilter decides it. staged is the open transaction’s view of the index.