Module search 

Search engine — Tantivy full-text search.

Provides BM25-scored full-text search over memory content. Index is stored alongside the LMDB store in a separate directory.

Recall-quality hygiene (see docs/TANTIVY_RECALL_QUALITY_FIX.md):

• Queries are stripped of common stopwords before parsing.
• Terms are only quoted when they contain reserved query syntax (plain terms — including hyphenated compounds — pass through so the tokenizer can split them into phrase matches).
• Content is sanitized at index time (binary/garbage content is skipped).
• Results are filtered by optional absolute and/or relative score floors, and output content is scrubbed of control characters.

Structs

IndexHealth

Tracked health of the Tantivy index relative to LMDB.

SearchEngine

The full-text search engine backed by Tantivy.

SearchOptions

Search options controlling recall behavior.

SearchResult

Search result item.

Constants

MAX_INDEX_CONTENT_LEN

Maximum content length (in chars) indexed into Tantivy.

MIN_PRINTABLE_RATIO

Minimum printable-char ratio for content to be indexed (0.9 = max 10% garbage).

STOPWORDS

Common English stopwords stripped from queries before parsing.

Functions

printable_ratio

Printable-character ratio used by the index/admission gate.

sanitize_content_for_index

Prepare content for indexing.

sanitize_tantivy_query

Sanitize a user-provided query string for Tantivy’s query parser.

scrub_text

Scrub text for output: replace control characters (except newline, tab, carriage return) with a space, and cap the length at MAX_INDEX_CONTENT_LEN.

strip_stopwords

Strip common English stopwords from a query string.

token_coverage

Fraction of a query’s stopword-stripped terms present in content (stemming-aware).