Crate sift_core 

Composable indexed code search engine.

sift-core builds on-disk indexes over codebases and uses them to narrow candidate files before running the full regex engine. The index layer is designed for multiple coexisting index types: each type independently narrows candidates, and the Indexes registry intersects their results.

Today the shipped index type is a trigram index (TrigramIndex), which records overlapping 3-byte sequences and achieves up to 60x speedup over ripgrep on indexed queries. Additional index types (AST indexes, dependency graphs, vector indexes) can be added by extending the IndexKind and Index enums.

Architecture

IndexStore::build(kinds) -> snapshot with index artifacts
Indexes::open(sift_dir)  -> registry of opened indexes
QueryPlanner::candidates -> intersect index candidate sets
SearchQuery::run          -> regex scan over narrowed candidates

Walking: WalkBuilder from the ignore crate.

Re-exports

pub use grep::GrepRun;

pub use candidate::Candidate;

pub use search::SearchError;

pub use search::SearchOutcome;

pub use search::SearchQuery;

pub use query::CandidateRequirement;

pub use query::QueryFlags;

pub use query::QueryPlanner;

pub use query::QuerySpec;

pub use query::UnindexedStrategy;

Modules

candidate

grep

Grep pipeline orchestration.

query

search

Indexed search execution built on the public grep crates.

Structs

CorpusMeta

Which corpus this store indexes.

CorpusSpec

Description of a corpus to index.

FileId

FilterMeta

Ignore and visibility rules in effect when the index was built.

IndexConfig

Configuration for building or updating an index over a corpus.

IndexId

IndexStore

Index lifecycle orchestrator backed by a [SnapshotStore] for atomic persistence and coordination.

IndexWalkConfig

Filesystem walk behavior for index builds and updates.

Indexes

Registry of opened indexes read from a snapshot store.

QueryPlanOutput

StoreMeta

Persistent store manifest (.sift/meta.json).

TrigramIndex

Opened trigram index with memory-mapped posting lists.

Walk

Walk is a recursive directory iterator over file paths in one or more directories.

WalkBuilder

WalkBuilder builds a recursive directory iterator.

WalkMeta

Filesystem walk behavior used when the index was built.

Enums

CorpusKind

Whether the index was built from a directory or a single file.

Error

Top-level umbrella error for all core operations.

Index

An opened index instance, used for query-time candidate narrowing.

IndexError

Errors specific to the index registry layer.

IndexKind

Tag identifying an index kind for lifecycle dispatch (build, open, update).

PlanMode

How an index query plan resolves candidates.

TrigramIndexError

Errors specific to opening or persisting a trigram index.

Constants

FILES_BIN

LEXICON_BIN

POSTINGS_BIN

SIFT_DIR

TRIGRAMS_BIN

Type Aliases

Result