brainwires-rag 0.11.0

Codebase indexing + hybrid retrieval (vector + BM25) for the Brainwires Agent Framework. Includes AST-aware chunking via tree-sitter (12 languages), Git history search, and reranking via spectral diversity / cross-encoder.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

//! Document processing, chunking, and hybrid search
//!
//! Provides capabilities for ingesting non-code documents (PDF, DOCX, Markdown, plain text),
//! chunking them into searchable pieces, and performing hybrid search (vector + BM25).
//!
//! ## Key Components
//!
//! - **DocumentProcessor** - Text extraction from various file formats
//! - **DocumentChunker** - Intelligent chunking respecting natural boundaries
//! - **DocumentStore** - Main API for indexing and searching documents
//! - **DocumentBM25Manager** - Per-scope BM25 keyword search
//! - **DocumentMetadataStore** - Document-level metadata persistence

pub mod bm25;
pub mod chunker;
pub mod lance_tables;
pub mod metadata_store;
pub mod processor;
pub mod store;
pub mod types;

// Re-export key types
pub use bm25::{DocumentBM25Manager, DocumentBM25Result, DocumentBM25Stats, document_rrf_fusion};
pub use chunker::DocumentChunker;
pub use metadata_store::DocumentMetadataStore;
pub use processor::DocumentProcessor;
pub use store::{DocumentScope, DocumentStore};
pub use types::{
    ChunkerConfig, DocumentChunk, DocumentMetadata, DocumentSearchRequest, DocumentSearchResult,
    DocumentType, ExtractedDocument,
};