Struct FullTextIndex 

pub struct FullTextIndex { /* private fields */ }

Production-quality full-text search index

This implementation provides enterprise-grade search capabilities with:

• Proper BM25 relevance scoring with document length normalization
• Efficient fuzzy search using Finite State Transducers
• Thread-safe concurrent access
• Memory-optimized data structures

Implementations

impl FullTextIndex

pub fn new(_collection: &str, field: &str) -> Self

Create a new full-text search index

Arguments

• collection - Name of the collection to index
• field - Name of the field within documents to index

Examples

let index = FullTextIndex::new("products", "description");

pub fn with_options( collection: &str, field: &str, k1: f32, b: f32, enable_stop_words: bool, ) -> Self

Create a search index with custom BM25 parameters

Arguments

• collection - Name of the collection to index
• field - Name of the field within documents to index
• k1 - Term frequency saturation parameter (typically 1.2-2.0)
• b - Document length normalization (0.0-1.0, where 0.75 is standard)
• enable_stop_words - Whether to filter common words

Examples

// Create index optimized for short documents (titles, names)
let index = FullTextIndex::with_options("products", "name", 1.6, 0.3, true);

// Create index optimized for long documents (articles, descriptions)
let index = FullTextIndex::with_options("articles", "content", 1.2, 0.85, true);

pub fn index_document(&self, doc: &Document) -> Result<()>

Add or update a document in the index with proper BM25 scoring

Arguments

• doc - The document to index

Returns

A Result indicating success or an error

Examples

index.index_document(&document)?;

pub fn search(&self, query: &str) -> Vec<(String, f32)>

Search with proper BM25 relevance scoring

Arguments

• query - The search query text

Returns

A vector of (document_id, relevance_score) pairs, sorted by relevance

Examples

let results = index.search("wireless bluetooth headphones");
for (doc_id, score) in results {
    println!("Document {}: relevance {:.3}", doc_id, score);
}

pub fn fuzzy_search( &self, query: &str, max_distance: usize, ) -> Vec<(String, f32)>

Efficient fuzzy search using Finite State Transducers

This implementation uses FST for O(log n) fuzzy matching instead of the naive O(n) approach of checking every term in the index.

Arguments

• query - The search query text
• max_distance - Maximum edit distance for fuzzy matching

Returns

A vector of (document_id, relevance_score) pairs

Examples

// Find documents matching "wireless" with up to 2 typos
let results = index.fuzzy_search("wireles", 2);

pub fn remove_document(&self, doc_id: &str)

Remove a document from the index efficiently

This optimized version doesn’t scan the entire index. Instead, it re-tokenizes the document and removes only the relevant entries.

Arguments

• doc_id - ID of the document to remove

Examples

index.remove_document("doc-123");

pub fn highlight_matches(&self, text: &str, query: &str) -> String

Highlight search terms in text

Arguments

• text - The text to highlight
• query - The search query

Returns

Text with search terms wrapped in tags

Examples

let highlighted = index.highlight_matches(
    "This is a wireless bluetooth device",
    "wireless bluetooth"
);
// Returns: "This is a <mark>wireless</mark> <mark>bluetooth</mark> device"

pub fn set_stop_words_filter(&mut self, enable: bool)

Configure stop word filtering

pub fn add_stopwords(&mut self, words: &[&str])

Add custom stop words

pub fn clear_stopwords(&mut self)

Clear all stop words

pub fn get_stopwords(&self) -> Vec<String>

Get current stop words

pub fn get_stats(&self) -> IndexStats

Get index statistics

pub fn get_fuzzy_matches( &self, query: &str, max_distance: u32, ) -> Vec<(String, f32)>

Get fuzzy matches for a term (legacy method for compatibility)