Skip to main content

Crate trueno_rag

Crate trueno_rag 

Source
Expand description

Trueno-RAG: Pure-Rust Retrieval-Augmented Generation Pipeline

This crate provides a complete RAG stack built on Trueno compute primitives with zero Python/C++ dependencies.

§Quick Start

use trueno_rag::{
    pipeline::RagPipelineBuilder,
    chunk::RecursiveChunker,
    embed::MockEmbedder,
    rerank::NoOpReranker,
    fusion::FusionStrategy,
    Document,
};

// Build a RAG pipeline
let mut pipeline = RagPipelineBuilder::new()
    .chunker(RecursiveChunker::new(512, 50))
    .embedder(MockEmbedder::new(384))
    .reranker(NoOpReranker::new())
    .fusion(FusionStrategy::RRF { k: 60.0 })
    .build()
    .unwrap();

// Index a document
let doc = Document::new("Machine learning enables computers to learn from data.")
    .with_title("ML Intro");
pipeline.index_document(&doc).unwrap();

// Query the pipeline
let results = pipeline.query("machine learning", 5).unwrap();
assert!(!results.is_empty());

§Chunking Strategies

Multiple chunking strategies are available:

§Fusion Strategies

Combine dense and sparse retrieval results:

§Example: Custom Chunking

use trueno_rag::{chunk::{ParagraphChunker, Chunker}, Document};

let chunker = ParagraphChunker::new(2); // 2 paragraphs per chunk
let doc = Document::new("Para 1.\n\nPara 2.\n\nPara 3.");
let chunks = chunker.chunk(&doc).unwrap();
assert_eq!(chunks.len(), 2);

Re-exports§

pub use chunk::Chunk;
pub use chunk::ChunkId;
pub use chunk::ChunkMetadata;
pub use chunk::Chunker;
pub use chunk::ChunkingStrategy;
pub use chunk::FixedSizeChunker;
pub use chunk::ParagraphChunker;
pub use chunk::RecursiveChunker;
pub use chunk::SemanticChunker;
pub use chunk::SentenceChunker;
pub use chunk::StructuralChunker;
pub use chunk::TimestampChunker;
pub use embed::Embedder;
pub use embed::EmbeddingConfig;
pub use embed::PoolingStrategy;
pub use embed::EmbeddingModelType;
pub use embed::FastEmbedder;
pub use error::Error;
pub use error::Result;
pub use fusion::FusionStrategy;
pub use index::BM25Index;
pub use index::SparseIndex;
pub use index::VectorStore;
pub use loader::transcription::TranscriptionBackend;
pub use loader::transcription::TranscriptionConfig;
pub use loader::transcription::TranscriptionLoader;
pub use loader::ImageLoader;
pub use loader::DocumentLoader;
pub use loader::LoaderRegistry;
pub use loader::SubtitleLoader;
pub use loader::TextLoader;
pub use media::parse_subtitles;
pub use media::SubtitleCue;
pub use media::SubtitleFormat;
pub use media::SubtitleTrack;
pub use metrics::AggregatedMetrics;
pub use metrics::RetrievalMetrics;
pub use pipeline::ContextAssembler;
pub use pipeline::RagPipeline;
pub use rerank::Reranker;
pub use retrieve::HybridRetriever;
pub use retrieve::RetrievalResult;
pub use sqlite::SqliteIndex;
pub use sqlite::SqliteStore;
pub use multivector::exact_maxsim;
pub use multivector::MockMultiVectorEmbedder;
pub use multivector::MultiVectorEmbedder;
pub use multivector::MultiVectorEmbedding;
pub use multivector::ResidualCodec;
pub use multivector::WarpIndex;
pub use multivector::WarpIndexConfig;
pub use multivector::WarpSearchConfig;
pub use retrieve::MultiVectorRetriever;

Modules§

chunk
Document chunking strategies for RAG pipelines
compressed
Compressed Index Serialization (GH-2)
embed
Embedding generation for RAG pipelines
error
Error types for Trueno-RAG
eval
Evaluation framework for RAG retrieval quality (PMAT-015)
fusion
Score fusion strategies for hybrid retrieval
index
Indexing for RAG pipelines (BM25 sparse index and vector store)
loader
Document loading abstraction for pluggable file format support.
media
Media types for subtitle and transcript support.
metrics
Retrieval evaluation metrics
multivector
Multi-vector retrieval with WARP algorithm
pipeline
RAG Pipeline implementation with context assembly
preprocess
Query preprocessing strategies for improved retrieval.
rerank
Reranking module for RAG pipelines
retrieve
Retrieval module for RAG pipelines
sqlite
SQLite+FTS5 persistent storage backend for RAG indices.

Structs§

Document
A document to be indexed
DocumentId
Document identifier

Enums§

Compression
Compression algorithm selector.