Expand description
ripindex — a persistent, memory-mapped inverted index over a directory tree.
Pipeline:
crawler ──► Vec<FileEntry> ──► index::builder ──► Index<MemTermStore>
(tokenizer) │
▼
query::parse ──► query::exec ──► Vec<Hit> ──► snippetStorage boundary. The query engine sees an index only through
index::IndexReader / index::SegmentReader and the posting-list
traits in index::postings. index::MemIndex is the in-RAM segment the
builder fills; store serialises it into the on-disk format in
docs/FORMAT.md and reads it back through memory maps behind the same
traits. Nothing in query/ or snippet knows which backend it is on.
File contents are never stored. Snippets re-read the file at query time.
Beyond a fresh build, store also covers reconciling an index
against a changed filesystem without rebuilding it (store::update_index,
backed by store::state’s staged, cheapest-first change detection),
merging small or heavily-tombstoned segments (store::merge_index), a
refreshable handle safe to hold across concurrent updates and merges
(store::LiveIndex), and a hint-driven watcher with deterministically
testable scheduling (store::watcher). The watcher is a hint, never a
source of truth — store::state’s module docs say why.
Re-exports§
pub use error::Error;pub use error::QueryError;pub use error::Result;pub use index::DocId;pub use index::Index;
Modules§
- bench
ripindex bench: build, measure open (cold, best-effort, and warm), time a set of queries, and watch RSS while they run.- crawler
- Directory crawler: turns a root directory into the ordered list of files the indexer will consider.
- daemon
- M4: the daemon. Every mutation to any root — watch, reconcile, update,
merge — happens in exactly one process, reached by every client (the CLI,
future plugins) over a local, permission-restricted transport. See
protocolfor the wire format,transportfor the socket/pipe layer,actorfor the per-root single-writer task,serverfor the accept loop, andclientfor the autostart-and-connect logic the CLI uses. - error
- format
- On-disk encodings — pure functions over byte slices, no I/O.
The layout is specified field-by-field in
docs/FORMAT.md. - fs
- File-system access behind a trait, so the commit protocol can run under injected faults.
- index
- Index abstractions and the in-memory segment.
- query
- Query parsing, evaluation and ranking.
- report
- Human-readable formatting shared by
indexandbench. - snippet
- Result snippets. The index holds no content, so the file is re-read here.
- store
- The persistent index:
.ripindex/under the indexed root, committed through an atomic manifest protocol. Layout and encodings are specified indocs/FORMAT.md;writerimplements §7 (commit),reader§8 (open). - tokenizer
- Unicode-aware tokenizer with code-identifier splitting.