sift-core

Indexed grep-style search engine. Build on-disk indexes, then run regex or fixed-string queries with automatic candidate narrowing. The shipped index type is a trigram index; the SearchIndex trait allows plugging in additional index kinds.

Modules

API

use sift_core::{SearchOptions, SearchQuery, TrigramIndex, TrigramIndexBuilder, Indexes, CandidateFilter};

// Build (using the shipped trigram index)
let index = TrigramIndexBuilder::new(&corpus_root).with_dir(&index_dir).build()?;

// Open
let index = TrigramIndex::open(&index_dir)?;

// Search
let indexes = Indexes::open(&sift_dir)?;
let search = SearchQuery::new(&patterns, SearchOptions::default())?;
let candidates = indexes.candidates(&query.spec(), coverage);
search.run(SearchExecution { candidates: &candidates, output, separators, collect_stats: false })?;

SearchQuery compiles the regex once; repeated run calls reuse the compiled matcher and searcher cache.

Features

Testing

Unit tests are co-located with implementation files in #[cfg(test)] mod tests blocks. Integration tests live in tests/.

cargo test -p sift-core
cargo bench -p sift-core --bench query
cargo bench -p sift-core --bench index
cargo bench -p sift-core --bench grep

See benches/README.md for the full benchmark and profiling workflow.