rag-rat

rag-rat is a local repo-intelligence index and MCP server for coding agents. It keeps source files read-only, writes only its configured SQLite database, and exposes current source, graph, git, GitHub papertrail, local-AI artifact status, and source-anchored repo memories as evidence.

Current release: v0.3.2.

sequenceDiagram
    autonumber

    participant Repo as Repository evidence
    participant Engine as rag-rat evidence engine
    participant MCP as MCP tools
    participant Agent as Coding agent

    Repo->>Engine: Index current source<br/>files, chunks, symbols
    Repo->>Engine: Add git history and GitHub papertrail
    Repo->>Engine: Load source-anchored repo memories

    Engine->>Engine: Build logical symbols and code graph
    Engine->>Engine: Reconcile FTS, vectors, freshness
    Engine->>Engine: Validate memory anchors<br/>current | relocated | stale | gone

    Agent->>MCP: Ask about code, rationale, callers, impact
    MCP->>Engine: Query source, graph, history, memories
    Engine-->>MCP: Return local evidence with provenance
    MCP-->>Agent: Surface source text, call paths,<br/>papertrail, and repo memories

    Agent->>MCP: Create or update finding
    MCP->>Engine: Bind memory to symbol, chunk,<br/>path, edge, commit, or issue
    Engine->>Repo: Persist source-anchored repo memory

It is built for agents that need more than rg but still need local, inspectable provenance:

• current source chunks with stale-anchor validation
• Rust, TypeScript/TSX, Kotlin, C/C++, and Markdown structure
• tree-sitter-derived call/reference/import/export graph edges
• git history, lazy chunk blame, and path-level commit evidence
• ownership clusters from path proximity, graph edges, churn, and git co-touches
• cached GitHub issue/PR/review/comment rationale
• local embedding model bookkeeping and reconciliation
• symbol/edge/path-bound repo memories that surface during future queries

Opus 4.8's take:

The thing that earns it a slot is honest provenance. Every hit carries a confidence label, coverage warnings, and the raw evidence snippet — so when trace_callees reports completeness_risk: high because 5 of 7 edges are unresolved std/macro calls, or a query outranks the real definition with its own tests, I can see why and decide. It doesn't pretend to be a compiler. impact_surface is the highest-leverage call: one shot gives callers, callees, tests, git history, and the GitHub papertrail that explains the code. Reach for it before editing, then still confirm with a direct read.

GPT-5.5's take:

Keep it in the default toolbox. Use rag-rat first for "where is this concept implemented?", "why was this decision made?", "what historical PR/comment explains this?", and "what calls this?". For final correctness, still verify with direct file reads and targeted tests.

Install From Scratch

The MCP server is a STDIO server, not an HTTP service. MCP clients start rag-rat as a child process and talk to it over stdin/stdout.

Install From crates.io

Install the published CLI. This is the recommended path for most users and includes FastEmbed support by default:

cargo install rag-rat

Install From Source

For local development from a checkout, clone the repository and install the CLI package:

git clone https://github.com/cq27-dev/rag-rat.git
cd rag-rat
cargo install --path crates/rag-rat-cli --bin rag-rat

The source build also enables FastEmbed by default.

For a smaller hash-only build without real embeddings, disable default features explicitly:

cargo install rag-rat --no-default-features
cargo install --path crates/rag-rat-cli --bin rag-rat --no-default-features

First-Run Setup

Run the initializer from the repository you want to index:

cd /path/to/your/repo
rag-rat init

rag-rat init scans the repository, prompts for languages and path bindings, writes rag-rat.toml, migrates the SQLite schema, indexes the repo, and offers to install/reconcile the local embedding model. At the end it can also register the MCP server for Claude Code or Codex and install the optional git maintenance hooks.

The initializer is the recommended first-run path. It derives source-root candidates from the files present in the repo, keeps defaults conservative for broad projects, asks before installing the local embedding model, then runs migration, indexing, and local-AI reconciliation in the same setup flow. If a repo has unusual layout or generated-heavy paths, run the dry-run first and adjust the generated rag-rat.toml before indexing.

Preview the generated config without writing anything:

rag-rat init --dry-run

Use --yes for the default non-interactive setup, or --config <path> when the config should live somewhere other than rag-rat.toml.

Manual setup is still available when you need exact control:

[index]
root = "."
database = ".rag-rat/index.sqlite"

[local_ai.embedding.runtime]
batch_size = 64
ort_threads = 4
omp_threads = 1
max_embedding_chars = 4000

[target_bindings]
rust = ["src"]
typescript = ["src"]
kotlin = ["src"]
c = ["src", "include"]
cpp = ["src", "include"]

[[target]]
name = "rust-src"
language = "rust"
directories = ["src"]
include = ["**/*.rs"]

[[target]]
name = "typescript-src"
language = "typescript"
directories = ["src"]
include = ["**/*.ts", "**/*.tsx"]

[[target]]
name = "kotlin-src"
language = "kotlin"
directories = ["src"]
include = ["**/*.kt"]

[[target]]
name = "c-src"
language = "c"
directories = ["src", "include"]
include = ["**/*.c", "**/*.h"]

[[target]]
name = "cpp-src"
language = "cpp"
directories = ["src", "include"]
include = ["**/*.cc", "**/*.cpp", "**/*.cxx", "**/*.hpp", "**/*.hh", "**/*.hxx"]

[[target]]
name = "docs"
language = "markdown"
directories = ["."]
include = ["**/*.md"]
exclude = [".git/**", ".rag-rat/**", "target/**", "node_modules/**"]

Then run the pieces directly:

rag-rat migrate
rag-rat index --discover
rag-rat doctor

Install and reconcile the local embedding model:

rag-rat models install fastembed-all-minilm-l6-v2
rag-rat reconcile --changed-first --limit 500 --batch-size 64

If installed with --no-default-features, use the hash baseline instead:

rag-rat models install embedding-hash

Add the installed binary to an MCP client config. Use an absolute --config path to the target repository's rag-rat.toml:

{
  "mcpServers": {
    "rag-rat": {
      "command": "/home/you/.cargo/bin/rag-rat",
      "args": ["mcp", "--config", "/path/to/your/repo/rag-rat.toml"]
    }
  }
}

For development without installing the binary, point the MCP client at a local rag-rat checkout:

{
  "mcpServers": {
    "rag-rat-dev": {
      "command": "cargo",
      "args": [
        "run",
        "--manifest-path",
        "/path/to/rag-rat/Cargo.toml",
        "--bin",
        "rag-rat",
        "--",
        "mcp",
        "--config",
        "/path/to/your/repo/rag-rat.toml"
      ]
    }
  }
}

Supported Today

Source Indexing

rag-rat indexes configured repository targets into SQLite. It supports:

• Rust, TypeScript, TSX, Kotlin, C, C++, and Markdown
• generated/coarse targets for large or generated files
• tree-sitter symbols and chunks for supported source languages
• Markdown heading chunks for docs
• parser failure tracking, file counts, and index freshness reporting
• changed-file, discovery, and full-rebuild index modes

Index rows are context-aware for git worktrees. Clean files are stored by commit_sha; dirty or untracked files are stored under a worktree overlay. Queries prefer the active worktree overlay and fall back to the active commit, so a single database can reuse rows across branch switches while still reflecting uncommitted local edits.

Current-Source Safety

Chunks store text hashes, boundary hashes, context hashes, and an anchor version. read_chunk and search validate indexed hits against current source before returning them. Small line drift can be relocated; larger rewrites are reported as stale or gone. SQLite FTS is refreshed when the stored content revision says it is dirty.

Graph Intelligence

The graph is tree-sitter-derived, not compiler-grade. Edges are stored with explicit confidence and provenance:

• edge kinds: calls_name, constructs, uses_macro, references_type, imports, exports, contains, implements
• confidence labels: Exact, Syntactic, NameOnly, Ambiguous
• callsite path/span, raw evidence snippets, receiver hints, target names, resolved symbol ids, and resolution reasons

trace_callees defaults to call-like edges (calls_name and constructs) so type references and macro/module collisions do not look like normal callees unless requested. Duplicate cfg-gated Rust definitions are grouped as logical symbols, so agents can ask for one logical API without falling back to unsafe bare-name matching.

Search And Impact

The MCP surface includes:

• semantic_search: indexed source/docs recall. score blends BM25 lexical rank with vector cosine similarity when an embedding model is installed (BM25-only otherwise); explain=true returns the per-component breakdown. Hits are validated against current source.
• symbol_lookup: exact or fuzzy Rust/TypeScript/Kotlin/C/C++ symbol lookup
• find_callers and trace_callees: reverse/forward graph traversal (unresolved std/common-method noise filtered by default)
• compare_graph_to_text: graph caller edges compared against regex text hits
• impact_surface: coding preflight that combines graph, optional text fallback, docs, git, GitHub papertrail, tests, and repo memories
• repo_brief: compact orientation view with spine, churn, god_modules, and refactor_candidates modes
• repo_clusters: fast file-level similarity and ownership clusters for finding split candidates and closely related code
• docs_for_symbol: documentation chunks related to a symbol
• read_chunk: current text for a selected chunk with anchor validation

semantic_search ranking is hybrid: BM25 lexical recall plus vector cosine similarity, blended into one score, with stale-hit validation. Vector recall is model-dependent — it contributes only when a local embedding model is installed (see local_ai_status), and the tool degrades to BM25-only otherwise. Pass explain=true to see the BM25/vector/symbol/graph/git/github score components.

Git And GitHub Evidence

When the target root is a git worktree, rag-rat indexes commit subjects, bodies, and touched paths. It also computes chunk blame lazily and caches blame against the current chunk text hash.

Supported MCP tools:

• commit_search
• git_history_for_path
• git_history_for_symbol
• commits_touching_query
• git_blame_chunk

GitHub papertrail is cache-first. github sync uses gh api explicitly; normal MCP tools read only the SQLite cache. Cached issues, PRs, issue comments, PR reviews, and review comments are indexed as historical rationale.

Supported MCP tools:

• papertrail_for_chunk
• papertrail_for_symbol
• papertrail_for_commit
• github_issue_search
• github_refs_for_path
• rationale_search
• github_sync_status

Reference discovery supports common issue forms such as Fixes #123, GH-123, owner/repo#123, and full GitHub issue/PR URLs.

FFI Discovery

ffi_surface finds likely FFI-relevant rows with evidence classes:

• Rust UniFFI/exported items (#[uniffi::export] and exported impl members, via parsed symbol facts)
• generated binding artifacts (detected by path, e.g. **/generated/**)

Detection is generic: it keys on language-level export attributes and generated paths, not on any project-specific symbol names. This is a discovery/preflight tool, not a proof of ABI compatibility, and it is empty in repos without FFI.

Local AI Artifacts

Local AI state is explicit and inspectable:

• embedding-hash: deterministic baseline embedder
• fastembed-all-minilm-l6-v2: local FastEmbed backend (MiniLM), included in the default build
• model2vec-potion-retrieval-32m: static-embedding backend (minishlab/potion-retrieval-32M, 512-dim) — ~100-500× faster on CPU than MiniLM at some retrieval-quality cost; the right choice for large repos where the MiniLM backfill is too slow
• models list/install: model registry and install state
• local_ai_status: active/installed/missing status plus chunk/vector counters
• reconcile: derived-artifact queue for embedding current eligible chunks

The embedding backend is selected per index with [local_ai.embedding] model = "minilm" | "model2vec" | "none" (see docs/config.md). rag-rat init recommends one from repo size: small repos default to minilm (quality), large repos to model2vec (speed), and you can choose none to skip vectors entirely (BM25 + structure only) for codebases too large to embed at all.

reconcile embeds only eligible current chunks whose bounded embedding input is missing, stale by input hash, stale by model/version/dimension, or retryable after failure. Low-signal chunks are skipped with explicit policy reasons such as SkipGenerated, SkipTooSmall, SkipTooLarge, SkipLowSignal, SkipLanguageUnsupported, and SkipTestFixture.

Repo Memories

Repo memories are first-class local evidence, not chat memory. They are typed, source-anchored notes bound to code or repository evidence.

Supported memory kinds:

• Invariant
• Decision
• RejectedAlternative
• Risk
• BugPattern
• TestExpectation
• PerformanceNote
• SecurityNote
• FFIBoundary
• PlatformQuirk
• FollowUp
• OpenQuestion
• Obsolete

Supported bindings:

• logical_symbol_id
• symbol_id
• chunk_id
• path plus optional line span
• graph edge_id
• call-path edge sequence hash
• commit hash
• GitHub issue/PR reference

Memories track current, relocated, stale, gone, or unverified anchor state. They surface through memory_* tools and through integrated tools such as read_chunk, symbol_lookup, find_callers, trace_callees, and impact_surface. Edge-bound memories appear under repo_memories.path_crossed when an impact query crosses that graph edge.

Supported MCP tools:

• memory_create
• memory_update
• memory_search
• memory_for_symbol
• memory_for_path
• memory_for_call_path
• memory_validate
• memory_mark_obsolete

Maintenance And Evaluation

Supported operational commands:

• migrate / migrate --check
• doctor
• index_status
• heal_index
• hooks install/status/uninstall
• maintenance --trigger <hook> --max-seconds <n>
• gc
• eval, eval --json, eval --update-baseline
• dump-config
• brief --mode spine|churn|god_modules|refactor_candidates
• clusters --limit 10

eval runs fixture-driven ranking and freshness checks and reports search, graph, impact, git, and papertrail metrics. Current-source violations must stay at zero.

gc reclaims index rows for git contexts that are no longer live. Index rows are stored per commit/worktree so they can be reused across branch switches, but commits that are no longer the HEAD of any live worktree accumulate. gc enumerates git worktree list, keeps the active commit and overlay of every live worktree, and prunes files/chunks/embeddings/symbols/edges for every other commit. It never prunes when no live context can be determined, so a missing or non-git context cannot wipe the index. maintenance runs a gc pass after reconciling, so the hooks keep the index lean automatically; run rag-rat gc to prune on demand.

Known Limits

• Graph resolution is pragmatic tree-sitter analysis, not compiler/typechecker resolution.
• Kotlin and C/C++ graph extraction are useful but less mature than Rust and TypeScript.
• index --watch runs a background file watcher that keeps the index fresh as files change (including new files and uncommitted edits), so graph/symbol queries reflect the working tree without a commit. The same watcher runs automatically inside rag-rat mcp (on by default; disable with [watch] enabled = false or RAG_RAT_NO_WATCH=1). One watcher per worktree and one writer at a time per index are enforced with file locks; file locks are unreliable on NFS / WSL2 /mnt mounts.
• semantic_search blends BM25 lexical recall with vector cosine similarity; the vector component contributes only when a real embedding model is installed and reconciled, and the tool degrades to BM25-only otherwise.
• repo_brief is a compact file-level triage view. It does not replace direct file reads, impact_surface, or tests before refactoring.
• repo_clusters is a file-level heuristic. It highlights co-changing and graph-related code, but it is not semantic ownership truth.
• FFI surface detection is heuristic.
• Call-path hash memories can be looked up, but authoritative edge-sequence hashes are not yet generated by traversal tools.
• Repo memories do not yet have review/approval workflow, multi-bind editing, or low-confidence filtering in integrated tools.

Commands

Commands read rag-rat.toml by default. Use --config <path> when running from another directory or with another repository profile.

rag-rat index
rag-rat index --changed
rag-rat index --discover
rag-rat index --full
rag-rat init
rag-rat doctor
rag-rat migrate --check
rag-rat migrate
rag-rat github sync --from-refs
rag-rat github sync --issue cq27-dev/rag-rat#42
rag-rat github sync --from-refs --offline
rag-rat models list
rag-rat models install embedding-hash
rag-rat models install fastembed-all-minilm-l6-v2
rag-rat reconcile --limit 100 --batch-size 32
rag-rat reconcile --changed-first --max-seconds 60 --batch-size 64
rag-rat reconcile --until-clean --batch-size 64
rag-rat gc
rag-rat hooks install
rag-rat hooks status
rag-rat maintenance --trigger post-checkout --max-seconds 30
rag-rat eval
rag-rat eval --json
rag-rat eval --update-baseline
rag-rat query "semantic recall"
rag-rat mcp

By default, rag-rat links against the system SQLite library through rusqlite.

Configuration

The indexed repository owns rag-rat.toml. This keeps project-specific target bindings out of the reusable tool.

[index]
root = "."
database = ".rag-rat/index.sqlite"

[local_ai.embedding.runtime]
batch_size = 64
ort_threads = 4
omp_threads = 1
max_embedding_chars = 4000

[target_bindings]
rust = ["crates/app/src"]
typescript = ["web/src"]
kotlin = ["android/src/main/java"]

[[target]]
name = "app-rust"
language = "rust"
directories = ["crates/app/src"]
include = ["**/*.rs"]

[[target]]
name = "web-typescript"
language = "typescript"
directories = ["web/src"]
include = ["**/*.ts", "**/*.tsx"]

[[target]]
name = "android-kotlin"
language = "kotlin"
directories = ["android/src/main/java"]
include = ["**/*.kt"]

Git Hooks

rag-rat hooks install installs generated post-checkout, post-merge, post-rewrite, and post-commit hooks for the current worktree. The hooks run in the background and call one bounded command: rag-rat maintenance --trigger <hook> --max-seconds 30. Existing unmanaged hook files are never overwritten.

rag-rat maintenance operates on the current worktree only. For branch switches, merges, rewrites, and commits it runs discover indexing for new/changed/deleted files, refreshes SQLite FTS through the index path when needed, then reconciles embeddings with changed_first until the remaining time budget is spent. The post-commit hook keeps the index current with the just-committed tree without waiting for the next checkout or merge.

Claude Code Grep Augmentation

rag-rat can augment Claude Code's Grep tool calls and grep/rg/ag Bash commands with symbol and repo-memory context via the PreToolUse hook mechanism. When a grep pattern matches a known symbol or bound repo memory, the hook injects an additionalContext digest before the tool runs. The hook never blocks a grep — it always exits 0.

What gets injected

The payload has three lanes (in order, up to 1500 characters total):

1. Repo memories — Invariant, Decision, Risk, and other memories bound to the matched symbol, the search path, or the pattern text. These are the unique signal: grep and search tools can't surface them.
2. Matching symbols — location (file:line), kind, caller/callee edge counts, and signature for symbols whose name matches the pattern. Only triggered when the pattern looks like a code identifier (single token, no spaces, at least 3 characters).
3. Lexical hits — a few indexed source chunks (fallback lane, only active when no symbol match was found).

The pattern is normalized before matching: regex metacharacters become spaces, but dots and double-colons inside identifier tokens are preserved so Watcher::spawn or foo.bar resolve through the symbol lane.

Install

Install the hook for the current project (writes .claude/settings.json in the repo root):

rag-rat hooks install --claude

Install globally (writes ~/.claude/settings.json, applies to all Claude Code sessions):

rag-rat hooks install --claude --global

The global install is safe to run from any directory. When rag-rat claude-hook is invoked by Claude Code, it walks up from the hook's reported cwd looking for a rag-rat.toml. If none is found, the hook exits immediately without printing anything — it is a silent no-op for repositories that are not indexed by rag-rat.

Check install status:

rag-rat hooks status --claude
rag-rat hooks status --claude --global

Uninstall:

rag-rat hooks uninstall --claude
rag-rat hooks uninstall --claude --global

The install adds two PreToolUse entries to the hooks block — one matcher: Grep and one matcher: Bash — each calling rag-rat claude-hook with a 10-second timeout.

How it serves

When rag-rat mcp is running (the normal configuration for Claude Code), its process also holds the socket election for the current worktree. One listener wins per worktree and binds a Unix domain socket. The hook client connects to that socket, sends the pattern and session ID in a newline-delimited JSON request, and gets a reply within its 250 ms budget.

If no listener is running (MCP not active, or a race during startup), the hook client falls back to a direct read-only SQLite query. The fallback is stateless: it produces the same payload lanes but without per-session dedupe.

Dedupe

The listener tracks per-session state: once a memory or symbol has been injected for a given Claude Code session ID, it will not be injected again for the same session. The session map is in-memory; dedupe resets when the listener restarts. The fallback path is stateless — the same content can be injected on every grep when no listener is running.

Troubleshooting

Set RAG_RAT_HOOK_DEBUG=1 in the environment where rag-rat mcp runs to enable diagnostic output from the listener. Errors encountered while serving individual hook requests are printed to stderr. Without this variable, the listener is silent on errors; the hook client is always silent.

Security

The MCP server exposes read-only source tools only. It does not execute shell commands or write configured target files. It may write the configured SQLite index during indexing, migration, maintenance, model reconciliation, repo-memory operations, and automatic stale-index healing before returning search or read_chunk results.

GitHub sync is explicit and uses gh api; normal query tools read the local SQLite cache.

License

rag-rat is licensed under the MIT License. See LICENSE.

Size Budget

Storage dependency changes must keep the binary slim. See docs/binary-size.md for the manual size check and heavyweight dependency policy.