Expand description
Optional hybrid (dense + lexical) semantic search engine for TurboVault.
Host-agnostic: this crate does not depend on turbovault-core,
turbovault-vault, or turbovault-plugin-api, and performs no I/O of its
own beyond CPU-bound embedding. IndexEngine is fed note content and
hands back what to persist; a host owns reading notes, deciding what
changed, and storing the result. crates/plugins/turbovault-plugin-vector
is that host today, wiring this engine to the TurboVault plugin boundary.
See the crate README for the stack this is built on and why, and for credit to the prototype this design started from.
Re-exports§
pub use chunk::chunk_text;pub use chunk::content_hash;pub use chunk::diff_chunks;pub use config::VectorConfig;pub use dense::DenseIndex;pub use embedding::EmbeddingEngine;pub use embedding::Model2VecEmbedder;pub use engine::IndexEngine;pub use engine::IndexStats;pub use engine::SearchHit;pub use error::Result;pub use error::VectorError;pub use lexical::LexicalIndex;pub use router::FusedResult;pub use router::reciprocal_rank_fusion;pub use store::ChunkRecord;pub use store::IndexMeta;pub use store::NoteRecord;
Modules§
- chunk
- Splitting note text into chunks, and diffing one note’s chunk set against another by content hash.
- config
- Resolved vector-search configuration.
- dense
- Dense (ANN) side of hybrid search: an in-memory HNSW graph over chunk
embeddings, backed by
hnsw_rs. - embedding
- Text-to-vector embedding backends.
- engine
- Ties chunking, embedding, the dense index, and the lexical index together into one incrementally-updatable hybrid search engine.
- error
- Errors the vector engine returns.
- lexical
- Lexical (BM25) side of hybrid search, backed by the
bm25crate. - router
- Reciprocal Rank Fusion: combine two independently-ranked candidate lists
into one ranking, the way
crate::engine::IndexEnginecombines the dense and lexical sides of a hybrid search. - store
- Persisted schema for the index. This crate does not perform I/O itself
(see the crate-level docs for why); these types are what a caller
serializes (with
serde_json, matching the convention the plugin boundary already uses forEventCursor) into whatever storage it has, and whatcrate::IndexEngine::restorerebuilds engine state from.