ragrig 0.9.6

RAG framework for research and prototyping. Zero dependencies, hot-swap any agent at runtime, hybrid BM25+vector retrieval. Default build compiles with cargo build --release and nothing else.
Documentation

Ragrig — RAG framework for Research and Prototyping

ragrig logo

A terminal-based Retrieval-Augmented Generation system built around three independently swappable AI agents — Embed, Memory, and Chat — each behind a Rust trait that allows hot-swapping backends at runtime.

Designed for students. The default build compiles with zero external dependencies — no C++ toolchain, no cmake, no protoc. Install Rust, install Ollama, run cargo build --release, and you're done. The binary weighs ~15 MB and runs on any desktop OS.

  • Zero extra dependencies — default build is pure Rust; Ollama provides models at runtime
  • Trait-driven — every pipeline stage is a Box<dyn Trait>; add new backends (OpenAI, Anthropic, Groq, …) or document parsers without touching existing code
  • Hardware-aware — delegate heavy models to the cloud, run small models locally, or go fully offline with CPU-only Fastembed (compiled into the binary) (--features internal-embed)
  • Hot-swappable — switch chat, memory, or embedding engines mid-session without losing document index or conversation context
  • Token-efficient cloud usage — use a tiny local model for query rewriting and only send the final prompt + context to an expensive cloud API
  • Hybrid retrieval — BM25 full-text search fused with cosine vector similarity via Reciprocal Rank Fusion. Hot-swappable ranking algorithms (RRF, weighted linear, MMR diversity, LLM re-rank) let you experiment with retrieval strategies without changing your document index
  • Pluggable ranking — the Ranker trait decouples scoring from storage; compare Cosine vs. BM25 vs. RRF fusion on the same document set at runtime
  • Cross-platform — Linux, macOS, WSL, and Windows (MSVC / MinGW)

What is RAG?

Retrieval-Augmented Generation lets an LLM answer questions about documents it has never seen before. Instead of stuffing every document into the prompt (which would overflow the context window), RAG works in two phases:

  1. Indexing — Documents are split into overlapping chunks, each chunk is converted to a numeric embedding vector (a list of floats that captures its meaning), and both the text and its vector are saved in a vector store.

  2. Querying — When you ask a question, the same embedding model converts your query to a vector. The store finds the k most similar chunks (via cosine similarity, BM25 keyword matching, or a hybrid of both). Those chunks are injected into the LLM prompt alongside your question, so the model can ground its answer in your documents.

Ragrig wraps this pipeline in three swappable agents — Embed (vectorise), Memory (optionally rewrite the query for better retrieval), and Chat (generate the final answer). The diagram below shows how data flows through the system:

flowchart LR
    U[User query] --> M[Memory]
    M -->|rewritten query| E[Embed]
    E -->|query vector| V[(Vector Store)]
    V -->|top-k chunks| C[Chat]
    U -->|original query| C
    C -->|grounded answer| R[Response]

If you are new to these concepts, a more detailed introduction can be found at retrieval-augmented-generation on the Prompt Engineering Guide.

Quick Start

You need three things

  1. Rust — rustup.rs
  2. Ollama — ollama.com/download
  3. Three models (run these once):
ollama pull gemma2:latest           # chat
ollama pull nomic-embed-text        # embeddings
ollama pull qwen2.5:1.5b           # memory / query-rewriting

Install

cargo install ragrig

This downloads and compiles the latest release from crates.io. The binary ends up in ~/.cargo/bin/ragrig — make sure that directory is on your PATH.

Or build from source

git clone https://github.com/schmettow/ragrig
cd ragrig
cargo build --release
./target/release/ragrig --folder ~/Documents/papers

Index and query

ragrig --folder ~/Documents/papers

First launch indexes all PDFs, EPUBs, DOCXs, and HTMLs in the folder. Subsequent launches are instant — only changed files are re-indexed.

Query > What are the key findings about forced-choice paradigms?

Students: if you only have Rust and Ollama installed, you already have everything you need. The default build adds nothing else.

Model parameters

Special operations, like a context-aware pseudonymizer, require fine-tuning of model parameters (temperature, top_p, etc.) to get deterministic, reproducible output. Ragrig supports this at every level — CLI, REPL, and library API.

# From the command line:
ragrig --folder ~/Documents/papers --temperature 0.1 --seed 42

# Or hot-swap at runtime from the REPL:
Query > /chat temperature 0.1
Query > /chat seed 42
Query > /chat top_p 0.9
Query > /chat max_tokens 2048

In library code, pass a GenerationParams struct when building your agent:

use ragrig::{agents::ChatAgentSpec, GenerationParams};
use std::convert::TryFrom;

let agent = Box::<dyn ragrig::agents::Generator>::try_from(ChatAgentSpec::Ollama {
    model: "qwen3.5:9b".into(),
    params: GenerationParams {
        temperature: Some(0.1),  // near-deterministic
        seed: Some(42),          // reproducible runs
        ..Default::default()
    },
})?;

You can also use .try_into()? instead of .build()? — both ChatAgentSpec and EmbedderSpec implement TryFrom for their respective trait objects, so they integrate with Rust's standard conversion ecosystem.

See examples/pseudonymizer for a complete multi-turn pseudonymization loop that uses temperature: 0.1 to produce consistent, privacy-preserving transcript rewrites.

Three-Agent Architecture

Every pipeline stage is a trait object — swap any agent at runtime without losing your document index or conversation memory.

Documents (PDF/EPUB/DOCX/HTML)
    │
    ▼
chunkedrs — token-accurate splitting with overlap
    │
    ├── Embedder trait ──────────────────────────────────────────┐
    │   OllamaEmbedder       (local, nomic-embed-text)           │
    │   FastembedEmbedder    (CPU-only, Nomic-Embed-Text-v1.5)   │
    │   NoopEmbedder         (pure chat, no document search)     │
    │                                                             │
    ▼                                                             │
VectorStore trait ────────────────────────────────────────────────┤
    BruteForceStore   (pure Rust, MessagePack on disk)  ← default │
    LanceDbStore      (Arrow columnar, hybrid BM25+vector)        │
    │                                                             │
    ▼                                                             │
Query                                                                    
    │                                                                    
    ▼                                                                    
Memory strategy (MemoryStrategy trait) ← hot-swap: /memory
    RewriteMemory / TranscriptMemory                                   
    │                                                                    
    ▼                                                                    
Embed → VectorStore.search → Ranker → top-k chunks                   
    │                                                                    
    ▼                                                                    
Chat agent (Generator trait)           ← hot-swap: /chat                
    OllamaGenerator / DeepSeekGenerator                                   
    │                                                                    
    ▼                                                                    
Streamed response with retrieved context + conversation memory

Hybrid Search Tuning

The vector store uses Reciprocal Rank Fusion (RRF, k=60) to combine cosine vector similarity with BM25 full-text search. Two parameters control retrieval quality:

Parameter Default What it does
top_k 50 Maximum chunks injected into the prompt
similarity_threshold 0.04 Cosine pre‑filter — chunks with cosine < threshold are excluded from RRF fusion

Understanding the threshold:

  • The threshold operates on cosine similarity (range: 0.0–1.0).
  • RRF fusion produces scores in the 0.0–0.03 range (rank‑based, not similarity‑based). The trace output shows RRF scores, not cosine scores.
  • A threshold of 0.0 passes everything; 0.04 filters out chunks with negligible vector overlap while letting BM25 keyword matches through.
  • Values above ~0.05 will aggressively prune — use when you have high‑quality embeddings and want strictly semantic results.

Tune at runtime:

Query > /search                      # show current values
Query > /search topk 10              # fewer chunks, tighter context
Query > /search threshold 0.08       # stricter semantic filter

Hot-Swap Examples

Start with everything local, switch chat to cloud mid-session:

Query > /chat deepseek deepseek-chat sk-...
Chat agent swapped: Ollama (gemma2:latest) → DeepSeek (deepseek-chat)

Forgetful mode — ask Alice's name, then make her forget:

Query > My name is Alice
Assistant > Nice to meet you, Alice!

Query > /memory off
Memory disabled (was: Ollama qwen2.5:1.5b)

Query > What's my name?
Assistant > I don't know — you haven't told me yet.

Raw transcript — no query rewriting, test context-window pressure:

Query > /memory transcript
Memory strategy: rewrite → transcript

Query > What is a vector database?
Assistant > A vector database stores embeddings ...

Query > Can you summarize that?
# "that" is NOT rewritten — the raw transcript in the prompt
# provides context.  Good for testing how models handle growing
# context windows with full conversation memory appended.

Session persistence — exit, restart, and recall past context:

Query > What are random effects in meta-analysis?
Assistant > Random effects models assume that the true effect size
varies across studies, as opposed to a single fixed effect …

Query > /exit
# next day …

$ ragrig --folder ~/papers
Session: 1718400000

Query > /memory log
History diffusion: off → log

Query > What was I asking about yesterday?
# The chat prompt now includes the raw transcript of the previous
# session, so the model can pick up the thread without you
# repeating yourself.
Assistant > Yesterday you asked about random effects in
meta-analysis.  We discussed how they differ from fixed-effect
models …

Pure chat — no document search, no memory, cloud-only:

Query > /embed none
Query > /memory off
Query > /chat deepseek deepseek-v4-pro
Query > Explain quantum entanglement in one paragraph.

Switch embeddings to CPU-only (no network):

Query > /embed fastembed
Embedder swapped: Ollama (nomic-embed-text) → Fastembed (Nomic-Embed-Text-v1.5)

Experiment with ranking algorithms — same index, different retrieval:

Query > /search rank Cosine
Ranker set to Cosine.

Query > /search rank BM25
Ranker set to BM25.

Query > /search rank Weighted alpha 0.7
Ranker set to Weighted.

Query > /search rank MMR lambda 0.7 inner Cosine
Ranker set to MMR.

Query > /search rank LLM inner Cosine model qwen2.5:0.5b
LLM reranker using Ollama (qwen2.5:0.5b)
Ranker set to LLM.

Compilation Paths

Default — Zero extra dependencies (recommended)

cargo build --release

Binary: ~15 MB. Nothing to install beyond Rust itself. Uses a pure-Rust vector store (custom BM25 + cosine similarity + RRF fusion, persisted to MessagePack). Embeddings come from Ollama over HTTP at runtime.

This is the path we ship to students. It compiles without a C++ toolchain, cmake, or protoc — works on Windows, macOS, and Linux with zero platform friction.

Internal embeddings — Fastembed (CPU-only)

cargo build --release --features internal-embed

Binary: ~35 MB. Adds FastembedEmbedder — runs Nomic-Embed-Text-v1.5 on the CPU. Zero network overhead for embeddings. Needs a C compiler (gcc or cl.exe) at build time. Use /embed fastembed at runtime.

LanceDB backend (large collections)

cargo build --release --no-default-features --features lancedb,ollama-embed

Binary: ~88 MB. Adds Arrow C++, protobuf, and compression codecs. Requires cmake and protoc at build time. Faster hybrid search for collections with 100k+ chunks.

Feature flags

Flag Default Description
ollama-embed on Local embeddings via Ollama HTTP (no extra deps)
internal on Pure-Rust vector store (MessagePack + cosine + BM25)
internal-embed off In-process Fastembed embeddings (needs C compiler)
internal-generate off In-process Candle LLM — zero network inference
offline off Meta: enables internal + internal-embed + internal-generate
lancedb off LanceDB hybrid index (needs protoc, Arrow C++)
test-fixtures off Compile-time embedded test documents for downstream crates

Binary size (release)

Features Size Native deps
Default (ollama-embed, internal) ~15 MB None — pure Rust
+ internal-embed ~35 MB ONNX Runtime (prebuilt binary)
--features offline ~250 MB Candle + ONNX Runtime — fully offline
+ lancedb ~88 MB Arrow C++, protobuf, compression

The offline feature is a convenience meta-flag: --features offline compiles ragrig into a fully self-contained binary with no network dependencies — every component (chat, embeddings, vector store) runs locally in-process. Use offline-cuda, offline-metal, or offline-mkl for GPU-accelerated variants.

Requirements

Dependency When needed
Rust 1.94+ Build (always)
Ollama Runtime — provides chat, embed, and memory models
C compiler (gcc/cl.exe) Only with --features internal-embed
C++ toolchain, protoc, cmake Only with --features lancedb

Default build: Rust + Ollama. Nothing else.

Platform Setup

Linux / macOS / WSL

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh   # Rust
cargo build --release                                                # that's it

Windows

  1. Install Rust from rustup.rs (MSVC host triple, the default)
  2. Run cargo build --release

No extra tools needed. If you later want Fastembed (--features internal-embed), install the Visual C++ Build Tools (select "C++ build tools" workload).

Commands

Command Action
Any text RAG query against your document pool
/download <url> Download and ingest a document by URL
/get 1,2,3-4,8 Bulk-download papers from last search results
/search <query> Search Semantic Scholar
/arxiv <query> Search arXiv (no rate limits)
/refs [topic] Extract references from last RAG results
/chat <b> [model] [key] | context <N> Hot-swap chat engine, set context window
/embed <b> [model] | purge | index | topk <N> | threshold <F> Hot-swap embedding, clear store, re-index, tune search
/memory <b> [model] [key] | transcript | off | purge Hot-swap memory, raw-transcript mode, disable memory, or clear it
/prompt chat|rewrite <file> | reset Load custom system prompts
/parser pdf unpdf|sink|extract|internal | epub epub Hot-swap document parser per format
/log [off|error|warn|info|debug|trace] Show or change log verbosity at runtime
/profile save|show|load|list [name] Manage configuration profiles (see below)
/search rank <name> [key value]* Hot-swap chunk ranking algorithm
/help Show available commands
exit / quit End session

Runtime log levels

By default ragrig prints informational messages (info level) — agent swaps, indexing progress, and errors. You can change the verbosity at any time without restarting:

/log           # show current level
/log debug     # enable diagnostic output from the ragrig crate only
/log trace     # enable full pipeline observability (rewrite, chunks, tokens)
/log warn      # back to quiet operation

At trace level the REPL logs every pipeline stage:

[TRACE] Query: "What is RAG?"
[TRACE] Rewrite: "What is RAG?" → "retrieval augmented generation definition"
[TRACE] Retrieved 5 chunks (cosine threshold 0.040):
[TRACE]   [0.0164] intro.pdf — "Retrieval-Augmented Generation (RAG) is a..."
[TRACE]   [0.0141] survey.pdf — "RAG systems have become the standard..."
[TRACE] Context budget: 8192 tokens (~24576 chars)  |  Full prompt: 3142 chars (~1047 tokens)
[TRACE] Elapsed: Some(1.234s)

The initial level can also be set via the RUST_LOG environment variable:

RUST_LOG=debug ragrig --folder ~/papers

Profile management

Profiles let you save and reload complete ragrig configurations — chat model, embedding backend, chunk size, search tuning, parser selection, and more — as JSON files. Use them to switch between project contexts without memorising long CLI invocations.

At startup — load a profile and optionally override individual flags:

# Save a profile from the REPL first:
ragrig --folder ~/papers --model gemma2:latest --chunk-size 512 --top-k 20
> /profile save physics

# Reload it later — chunk-size and top-k come from the profile,
# but you can still override on the CLI:
ragrig --folder ~/papers --profile physics --model gemma4:e4b

In the REPL:

Command Action
/profile save <name> Serialise current config (including runtime hot-swaps) to .ragrig/profiles/<name>.json
/profile show [name] Pretty-print a profile as JSON. show current prints the live running state.
/profile load <name> Load a profile into memory without restarting agents — use /chat, /embed, /memory afterwards to apply it.
/profile list List all saved profile names.

Profiles are stored under <folder>/.ragrig/profiles/. The JSON is hand-editable if you prefer typing values over REPL commands.

CLI Flags

Usage: ragrig --folder <FOLDER>

Options:
  -f, --folder <FOLDER>            Document directory (PDFs, EPUBs, DOCXs, HTMLs)
  -p, --profile <NAME>             Load a saved profile from .ragrig/profiles/ (JSON)
      --provider <PROVIDER>        Chat backend: ollama (default) or deepseek
      --deepseek-api-key <KEY>     DeepSeek API key [env: DEEPSEEK_API_KEY]
      --deepseek-model <MODEL>     DeepSeek model [default: deepseek-v4-pro]
  -m, --model <MODEL>              Ollama chat model [default: gemma2:latest]
      --embedding-provider <P>     Embedding: ollama (default) or fastembed
  -e, --embedding-model <MODEL>    Ollama embedding model [default: nomic-embed-text]
      --memory-model <MODEL>      Memory/rewrite model [default: qwen2.5:1.5b]
      --prompt-chat <FILE>         Custom system prompt for chat agent
      --prompt-rewrite <FILE>      Custom system prompt for rewrite agent
      --pdf-parser <BACKEND>       PDF parser: unpdf (default), sink, extract, internal
  -t, --threads <N>                Worker threads [default: 4]
      --embedding-concurrency <N>  Concurrent embedding requests [default: 32]
      --chunk-size <TOKENS>        Max tokens per chunk [default: 1024]
      --chunk-overlap <TOKENS>     Overlap between chunks [default: 128]
      --top-k <N>                  Chunks per query [default: 50]
      --similarity-threshold <FL>  Cosine similarity pre‑filter [default: 0.04]
      --model-ctx-tokens <N>     Context window budget for prompt truncation [default: 4096]
      --semantic-scholar-api-key <K>  API key [env: SEMANTIC_SCHOLAR_API_KEY]

API Usage (Developers)

ragrig is a library. Build your own frontend — GUI, web server, headless bot — on top of the same traits.

use ragrig::{
    embed::{EmbedderSpec, OllamaEmbedder},
    agents::{ChatAgentSpec, Generator},
    parsers::{DocumentParsers, build_parsers},
    store::open_store,
    types::ChunkConfig,
    vector::{collect_documents, search_similar},
};
use std::path::Path;

// Build agents and parser registry
let embedder = EmbedderSpec::Ollama { model: "nomic-embed-text".into() }.build()?;
let chat_agent = ChatAgentSpec::Ollama { model: "gemma2:latest".into(), params: Default::default() }
    .build()?;
let parsers = DocumentParsers::new(build_parsers());
let folder = Path::new("./my_docs");
let chunk_cfg = ChunkConfig::default();
let store = open_store(folder).await?;

// Index documents
let _stats = collect_documents(&*embedder, &parsers, folder, &chunk_cfg, &*store).await?;

// Search
let results = search_similar(&*embedder, 5, 0.0, &*store, "quantum computing").await?;

// Chat
chat_agent.generate_stream(&prompt, &|token| { print!("{}", token); }).await?;

Configuration profiles

RagrigConfig is the library's single format-agnostic configuration surface. It composes four sub-configs (ChatConfig, EmbedConfig, ParseConfig, MemoryConfig) and derives Serialize + Deserialize — so the same struct works for CLI arguments, JSON, TOML, or programmatic construction.

Three entry points:

use ragrig::types::RagrigConfig;

// 1. Programmatic — builder-style via Default + struct update
let config = RagrigConfig {
    folder: "./my_docs".into(),
    chat: ChatConfig { model: "gemma4:e4b".into(), ..Default::default() },
    ..Default::default()
};

// 2. From a JSON file (zero extra dependencies — serde_json is already included)
let json = std::fs::read_to_string("profile.json")?;
let config: RagrigConfig = serde_json::from_str(&json)?;

// 3. From a TOML file (add `toml = "0.8"` to Cargo.toml)
let toml_str = std::fs::read_to_string("profile.toml")?;
let config: RagrigConfig = toml::from_str(&toml_str)?;

Example TOML profile — save this as profile.toml and load it at startup via a small wrapper binary, or deserialise it programmatically:

folder = "./research-papers"
semantic_scholar_api_key = "s2k-xxxxxxxxxxxx"

[chat]
provider = "Ollama"
model = "gemma2:latest"
context_tokens = 8192
context_size_mode = "Auto"

[chat.params]
temperature = 0.1
max_tokens = 2048

[embed]
provider = "Ollama"
model = "nomic-embed-text"
top_k = 20
similarity_threshold = 0.04

[parse]
pdf_parser = "Extract"
chunk_size = 512
chunk_overlap = 64

[memory]
model = "qwen2.5:1.5b"

Merging CLI overrides on top of a profile — use override_with() to apply only the fields the user explicitly changed on the command line:

// Load the base profile
let mut config: RagrigConfig = serde_json::from_str(&fs::read_to_string("profile.json")?)?;

// Parse CLI args and convert to a RagrigConfig (binary-side code)
let cli_config = RagrigConfig::from(Cli::parse());

// Merge: CLI values override profile values where they differ from defaults
config.override_with(&cli_config);

Adding a new backend

Implement the Generator, Embedder, VectorStore, or DocumentParser trait:

struct OpenAiChat { model: String, api_key: String }

#[async_trait]
impl Generator for OpenAiChat {
    async fn generate_stream(&self, prompt: &str, on_token: &(dyn Fn(String) + Sync)) -> Result<()> {
        // POST to https://api.openai.com/v1/chat/completions, stream SSE chunks
    }
    fn backend_name(&self) -> &'static str { "OpenAI" }
    fn model_name(&self) -> &str { &self.model }
}

Then wire it into ChatAgentSpec::parse("openai", ...) — no other code changes needed.

Implementing a new document parser

Add support for a new PDF backend or file format (~30 lines). Example using justpdf (pure-Rust PDF library):

use ragrig::parsers::DocumentParser;
use std::path::Path;

struct JustpdfParser;

impl DocumentParser for JustpdfParser {
    fn extensions(&self) -> &[&str] { &["pdf"] }

    fn parse(&self, path: &Path) -> anyhow::Result<String> {
        let bytes = std::fs::read(path)?;
        let doc = justpdf::Document::load(&bytes)?;
        let mut md = String::new();
        for page in doc.pages() {
            md.push_str(&page.text());
            md.push_str("\n\n");
        }
        Ok(md)
    }

    fn name(&self) -> &'static str { "justpdf" }
}

Then register it in parsers::build_parsers() (or hot-swap via /parser pdf justpdf once you add the variant to PdfParserBackend). The chunker, embedder, and search pipeline all work unchanged — they only see Markdown.

Implementing a custom memory strategy

Memory backends implement the [MemoryStrategy] trait. The trait controls only query rewriting — the session always replays the raw transcript whenever *a strategy is active, regardless of whether rewriting happened.

Example: a strategy that rewrites using only the immediately preceding turn, discarding older turns so the rewriter isn't distracted by stale context:

use async_trait::async_trait;
use ragrig::{agents::Generator, memory::MemoryStrategy};

struct LastTurnOnly {
    agent: Box<dyn Generator>,
}

#[async_trait]
impl MemoryStrategy for LastTurnOnly {
    async fn generate_rewrite(&self, prompt: &str) -> Option<String> {
        // The prompt is "Conversation:\nUser: …\nAssistant: …\n\n"
        // followed by the system rewrite prompt.  Split at the
        // double-newline, then grab only the last User/Assistant pair.
        if let Some((memory_part, rest)) = prompt.split_once("\n\n") {
            let lines: Vec<&str> = memory_part.lines().collect();
            let mut tail = Vec::new();
            for line in lines.iter().rev() {
                if line.starts_with("User: ") || line.starts_with("Assistant: ") {
                    tail.push(*line);
                    if tail.len() >= 2 {
                        break;
                    }
                }
            }
            tail.reverse();
            let trimmed = format!("Conversation:\n{}\n\n{}", tail.join("\n"), rest);
            self.agent.generate(&trimmed).await.ok()
        } else {
            None
        }
    }

    fn name(&self) -> &'static str { "last-turn" }
}

The trait provides three methods:

Method Purpose
generate_rewrite(prompt) -> Option<String> Return Some(rewritten) to replace the query before vector search, or None to use the raw query.
clear() Wipe persistent state (default no-op).
name() Label displayed in /memory output.

Built-in strategies (RewriteMemory, TranscriptMemory) cover the common cases; implement the trait directly when you need custom truncation, keyword extraction, or external rewriter services.

Implementing a custom history strategy

History backends implement the [HistoryStrategy] trait from ragrig::longterm_memory. The trait controls how past sessions are diffused into the current chat prompt.

Example: a strategy that loads only the most recent session, formats a compact summary header, and skips the full transcript:

use async_trait::async_trait;
use ragrig::longterm_memory::{HistoryStrategy, SessionStore};

struct LatestSessionOnly;

#[async_trait]
impl HistoryStrategy for LatestSessionOnly {
    async fn build_context(
        &self,
        store: &dyn SessionStore,
        current_query: &str,
    ) -> anyhow::Result<String> {
        let manifests = store.list().await?;
        let Some(latest) = manifests.last() else {
            return Ok(String::new());
        };
        let Some(session) = store.load(&latest.id).await? else {
            return Ok(String::new());
        };
        // Extract just the questions the user asked last session.
        let questions: Vec<&str> = session
            .turns
            .iter()
            .filter(|t| matches!(t.role, ragrig::TurnRole::User))
            .map(|t| t.text.as_str())
            .collect();
        Ok(format!(
            "[Last session ({:?}): {} turn(s), topics: {}]\n",
            session.created,
            session.turns.len(),
            questions.join("; "),
        ))
    }

    fn name(&self) -> &'static str {
        "latest-session-only"
    }
}

The trait provides two methods:

Method Purpose
build_context(store, query) -> String Return a preamble injected into the system prompt. Return "" to skip.
name() Label displayed in /memory output.

Built-in strategies (LogHistory, SummaryHistory) cover the common cases; implement the trait directly when you need custom filtering, selection from multiple sessions, or non-LLM recombination.

Implementing a custom ranker

The [Ranker] trait lets you plug in any scoring algorithm. Implement rank(), name(), and clone_box():

use ragrig::{Ranker, ScoredChunk, store::StoredChunk};

#[derive(Debug, Clone)]
struct LengthRanker;

#[async_trait]
impl Ranker for LengthRanker {
    async fn rank(
        &self,
        chunks: &[StoredChunk],
        _query_vec: &[f32],
        _query_text: &str,
        top_k: usize,
        _threshold: f64,
    ) -> Vec<ScoredChunk> {
        // Trivial baseline: prefer shorter chunks (less context bloat).
        let mut indexed: Vec<(usize, f64)> = chunks
            .iter()
            .enumerate()
            .map(|(i, c)| (i, -(c.text.len() as f64)))
            .collect();
        indexed.sort_by(|a, b| a.1.total_cmp(&b.1));
        indexed.truncate(top_k);
        indexed
            .into_iter()
            .map(|(idx, _)| ScoredChunk {
                score: 0.0.into(),
                chunk: chunks[idx].text.clone().into(),
            })
            .collect()
    }

    fn name(&self) -> &'static str { "length" }
    fn clone_box(&self) -> Box<dyn Ranker> { Box::new(self.clone()) }
}

The built-in rankers — HybridRrfRanker, WeightedFusionRanker, MmrDiversityRanker, and LlmReranker — already cover the most common retrieval strategies. The decorator pattern (MmrDiversityRanker and LlmReranker) lets you wrap any inner ranker for diversity or LLM re-ranking.

Test fixtures for downstream crates

Enable the test-fixtures feature to get compile-time embedded copies of ragrig's own test documents — PDF, R Markdown, and HTML files suitable for writing parser integration tests without shipping your own files.

# Cargo.toml
[dev-dependencies]
ragrig = { version = "0.5", features = ["test-fixtures"] }

use ragrig::fixtures;

#[test]
fn parse_all_pdf_fixtures() {
    // fixtures::pdf::DIR is an include_dir::Dir with all files baked in.
    for entry in fixtures::pdf::DIR.files() {
        let name = entry.path().to_str().unwrap();
        let tmp = std::env::temp_dir().join(name);
        std::fs::write(&tmp, entry.contents()).unwrap();

        let parsers = ragrig::DocumentParsers::new(ragrig::parsers::build_parsers());
        let markdown = parsers.parse(&tmp).unwrap();
        assert!(!markdown.is_empty(), "{} produced no text", name);

        let _ = std::fs::remove_file(&tmp);
    }
}

// Also available as named constants:
assert!(fixtures::rmd::GETTING_STARTED.len() > 1000);
assert!(fixtures::html::INDEX.len() > 100);

Reactive UI integration (egui, ratatui, web, …)

Streaming generation to a GUI or TUI is a 4-call pattern. The same slim API works identically in egui, ratatui, a web server (SSE), or any reactive framework:

use ragrig::agents::{ChatAgentSpec, Generator};
use tokio::sync::mpsc;

// 1. Build the agent — one line
let agent = ChatAgentSpec::Ollama { model: "gemma2:latest".into() }.build()?;

// 2. Run generation on a background runtime, bridge to UI via channel
let (tx, mut rx) = mpsc::unbounded_channel::<String>();
let agent = std::sync::Arc::new(agent);
let agent_clone = agent.clone();
tokio::runtime::Runtime::new()?.spawn(async move {
    let _ = agent_clone.generate_stream(&prompt, &|token| {
        let _ = tx.send(token);  // callback is sync, channel decouples
    }).await;
});

// 3. Drain tokens in the UI loop (called every frame / event loop tick)
fn poll_stream(rx: &mut mpsc::UnboundedReceiver<String>, buffer: &mut String) -> bool {
    loop {
        match rx.try_recv() {
            Ok(t)         => buffer.push_str(&t),   // more tokens coming
            Err(Empty)    => return false,           // nothing right now
            Err(Disconnected) => return true,        // generation done
        }
    }
}

That's it — 4 ragrig calls: build, spawn, generate_stream, try_recv. The remaining 95% of a chat UI is framework-specific layout and input handling, not ragrig. See examples/streaming_chat_egui/ and examples/streaming_chat_ratatui/ for complete runnable demos.

Typed errors

ragrig defines typed error variants in [RagrigError] that carry structured payloads so callers can recover programmatically:

Variant Payload Recovery
ContextSizeExceeded current, max (tokens) Reduce top_k or expand context window
EmbedModelNotFound model: String Run ollama pull {model} and retry
StoreCorrupt path: String Delete the store file and re-index
NoDocumentsFound folder: String Add PDF, EPUB, or HTML files to the folder
OllamaUnreachable context: String Start Ollama with ollama serve — see troubleshooting for common causes
GenerationFailed backend, model, detail Check model is pulled and fits in VRAM

Every variant provides a [suggested_action()] method with a human-readable recovery hint. Use [RagrigError::log_or()] to log the error and its action in one call.

Downcast from anyhow::Error and switch on the variant:

use ragrig::RagrigError;

let result = agent.generate_with_context("query", &[]).await;
match result {
    Err(e) => {
        if let Some(ce) = e.downcast_ref::<RagrigError>() {
            match ce {
                RagrigError::ContextSizeExceeded { current, max } => {
                    eprintln!("Prompt ({current} tk) exceeds model limit ({max} tk). Truncating.");
                }
                RagrigError::EmbedModelNotFound { model } => {
                    eprintln!("Run: ollama pull {model}");
                }
                RagrigError::StoreCorrupt { path } => {
                    std::fs::remove_file(path).ok();
                    eprintln!("Removed corrupt store. Re-index on next run.");
                }
                RagrigError::NoDocumentsFound { folder } => {
                    eprintln!("No supported files in {folder}. Add PDFs, EPUBs, or HTML.");
                }
            }
        } else {
            eprintln!("Unexpected: {e}");
        }
    }
    Ok(answer) => println!("{answer}"),
}

Runnable examples

Clone the repo and run any example with cargo run in its directory (an Ollama server must be running):

git clone https://github.com/schmettow/ragrig.git
cd ragrig

# Single-shot RAG query — index fixtures, search, generate
cargo run --manifest-path examples/rag_query/Cargo.toml -- "What is RAG?"

# Two-agent dialog with shared vector store and transcript
cargo run --manifest-path examples/dialog/Cargo.toml -- "What is a p-value?"

# Streaming chat GUI with markdown bubbles (egui)
cargo run --manifest-path examples/streaming_chat_egui/Cargo.toml

# Streaming chat TUI with two-color bubbles (ratatui)
cargo run --manifest-path examples/streaming_chat_ratatui/Cargo.toml

# Streaming chat GUI with chat bubbles, provider/model picker, and RAG folder (Iced)
cargo run --manifest-path examples/streaming_chat_iced/Cargo.toml

# Binary with embedded vector store — indexed at build time
cargo run --manifest-path examples/embedded_togo/Cargo.toml -- "What is RAG?"
Example Concept
rag_query Single-shot pipeline: index → embed → search → generate via RagAgent
dialog Multi-agent orchestration: two RagAgent instances sharing one vector store and one transcript
streaming_chat_egui Reactive GUI: generate_stream + channel bridge → egui markdown bubbles
streaming_chat_ratatui Reactive TUI: same channel pattern → ratatui two-color bubbles with scroll
streaming_chat_iced Reactive GUI: Iced native GUI with provider/model picker, RAG folder picker, and streaming chat bubbles
embedded_togo Embedded store: build.rs indexes fixtures at compile time, include_bytes! bakes it into the binary

Transcripts

The TurnPairs newtype converts a session's Vec<Turn> into a slice of (&str, &str) pairs suitable for RagAgent::generate_with_context():

use ragrig::{Turn, TurnRole, TurnPairs};

let turns = vec![
    Turn { role: TurnRole::User, text: "Hello".into(), perf: None },
    Turn { role: TurnRole::Assistant, text: "Hi!".into(), perf: None },
];
let pairs = TurnPairs::from(&turns[..]);
agent.generate_with_context("What is RAG?", &pairs.0).await?;

Q & A

What is unique about ragrig and why should I use it?

Ragrig tries to be a flexible and zero-friction prototyping tool for researchers and students, not an enterprise-grade framework with all bells and whistles. Here are the points that distinguish Ragrig from other crates:

Zero native dependencies in default build.** Every other crate needs at minimum a C compiler (for tokenizers, ONNX runtime, tree-sitter, etc.) or an API key. Ragrig builds with cargo build --release and nothing else. This is a genuinely unique selling point for students, workshops, and quick-start scenarios.

  1. Runtime hot-swapping via trait objects. Every other crate uses compile-time feature flags to select backends. Ragrig lets you switch chat/embed/memory engines mid-session without losing state. langchainrust has multiple providers but you pick them at Cargo.toml time. ragrig's /chat deepseek, /embed fastembed, /memory off commands have no equivalent in any competitor.

  2. Panic-safe multi-parser PDF pipeline. Three PDF parsers (pdfsink for layout-aware, pdf-extract for flat text, sloppy binary scavenger as fallback) with catch_unwind wrapping. No other crate does this — they pick one parser and crash on malformed PDFs.

  3. Token-efficient cloud usage pattern. Use a tiny local model for query rewriting, only send the final prompt + context to the cloud. This is described in the README hot-swap examples and baked into the MemoryStrategy trait. No competitor has this pattern explicitly designed in.

  4. Student-focused UX. The README's quick-start is 3 commands (rustup, ollama pull ×3, cargo build --release). The REPL has 15+ slash commands with clear transition messages. Session persistence works out of the box.

When should I not use it?

Ragrig is designed as an accessible framework to build multi-agent interactive prototypes. It is not intended for production use or highly scalable deployments. For these purposes, you should use a dedicated RAG framework like rig-core on which Ragrig is heavily based.

I am a Python programmer. I am not able to program in Rust. How can I use Ragrig?

Ragrig provides a fully documented API with numerous examples and a dedicated agent skill (only available on Github). With this information, a good coding agent can produce working Ragrig applications with not more than a few instructions.

For version 2.0, we plan to provide Python (and possibly R) bindings.

Ollama is unreachable — what should I check?

If ragrig reports OllamaUnreachable, work through these in order:

  1. Ollama isn't running. Start it in a terminal:

    ollama serve

    On macOS and Windows, launching the Ollama desktop app also starts the server.

  2. Ollama is running on a non-default port. By default ragrig connects to localhost:11434. If you changed the port (e.g. via OLLAMA_HOST), set the same variable in the terminal where ragrig runs:

    export OLLAMA_HOST=127.0.0.1:11435
cargo run -- --folder ./my_docs

  3. A model pull was interrupted. Partial downloads can leave the Ollama registry in a broken state. Re-pull the model:

    ollama pull nomic-embed-text
ollama pull gemma2:latest

    If that fails, remove the partial model and pull fresh:

    ollama rm nomic-embed-text && ollama pull nomic-embed-text

  4. Firewall or port conflict. Ensure port 11434 is not blocked by a firewall or already bound by another process:

    # Linux / macOS / WSL
lsof -i :11434
# Windows (PowerShell)
netstat -ano | findstr :11434

  5. WSL → Windows networking. When Ollama runs on Windows and ragrig runs inside WSL, localhost does not automatically forward. Find the Windows host IP from inside WSL and set OLLAMA_HOST:

    export OLLAMA_HOST=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}'):11434

    Alternatively, install Ollama directly inside WSL so both processes share the same network namespace.

When the context size exceeds the model's maximum, how can I adjust this?

Context-size errors happen for two reasons:

  1. Hardware VRAM limits — Ollama caps the context window at 4096 tokens on GPUs with less than 24 GB VRAM to prevent out-of-memory crashes.
  2. Architectural limits — some distilled reasoning models (e.g. DeepSeek R1 8B/14B) have a hard-coded 4096-token maximum that even Ollama cannot override.

Ragrig detects context overflows automatically. By default, when the model reports a [RagrigError::ContextSizeExceeded], the binary auto-adjusts its budget to the model's actual maximum, rebuilds the prompt with fewer chunks, and retries once. You see:

[INFO] Context overflow — shrinking budget to 9216 chars, retrying.

If the retry also fails, pass --context-size-forced to keep the original error path, then set a manual budget:

./target/release/ragrig --folder ~/papers --model-ctx-tokens 4096
# or mid-session:
Query > /chat context 4096

Library consumers can catch the typed error directly:

match chat_agent.generate(prompt).await {
    Err(e) => {
        if let Some(ce) = e.downcast_ref::<ragrig::RagrigError>() {
            // ce.current_size(), ce.max_size() — use these to trim
            // your embedding results before retrying.
        }
    }
    Ok(response) => { … }
}

License

MIT License — see LICENSE.