Crate semtree_rag 

RAG pipeline for code: index, search, and inject codebase context into LLM prompts - fully on-device, no daemon, no API key.

semtree-rag is the composable core behind the semtree CLI. It wires a pluggable Embedder and VectorStore into a full pipeline - parse → chunk → embed → index → search - that you can drop into your own tool, an MCP server, or an LLM context provider.

Pipeline at a glance

Stage	Building blocks
Index a directory	Indexer + ChunkRegistry
Search (vector / BM25 / fused)	HybridSearcher over SearchEngine + LexicalIndex
Build an LLM prompt	ContextBuilder → ContextWindow
Incremental re-index	FileManifest

Example

use std::sync::Arc;
use semtree_embed::fastembed::FastEmbedder;
use semtree_store::usearch::UsearchStore;
use semtree_rag::{ChunkRegistry, HybridSearcher, Indexer, LexicalIndex, SearchEngine, SearchMode};

// Backends are traits - swap FastEmbedder for OpenAI, UsearchStore for Qdrant.
let embedder = Arc::new(FastEmbedder::new()?);
let store = Arc::new(UsearchStore::new(384)?);

// Index: parse -> chunk -> embed -> store. The registry keeps chunk metadata.
let mut registry = ChunkRegistry::default();
Indexer::new(embedder.clone(), store.clone())
    .index_dir(std::path::Path::new("./src"), &mut registry, None, |_, _| {})
    .await?;

// Search: fuse vector similarity with BM25 keyword matching.
let engine = SearchEngine::new(embedder, store);
let lexical = LexicalIndex::from_chunks(registry.iter());
let hits = HybridSearcher::new(engine, lexical)
    .search("how is authentication handled", 5, SearchMode::Hybrid)
    .await?;

A complete, runnable version lives in examples/build_your_own.rs.

Structs

ChunkRegistry

Persists chunk metadata alongside the vector index.

ContextBuilder

ContextSnippet

ContextWindow

FileManifest

Tracks per-file state to enable incremental re-indexing.

HybridSearcher

Combines a vector SearchEngine with a LexicalIndex so queries can be matched by meaning, by keyword, or both (fused). Hybrid is what lets semtree catch concepts a grep misses and keep the exact-identifier precision a pure vector search loses.

Indexer

LexicalIndex

In-memory BM25 lexical index over indexed chunks.

SearchEngine

Enums

RagError

SearchMode

How a query is matched against the index.

Functions

collect_indexable_files

Collect all indexable files under dir, respecting the ignore list.