LeIndex

Semantic code intelligence for humans and AI agents — a Program Dependence Graph, hybrid neural search, and 18 structural tools behind one small daemon.

Ask LeIndex where authentication lives, what breaks if you rename a function, or how the indexer resumes across phases — and get the symbols, callers, and source back in milliseconds, not a dump of files to read.

you:     "where is request validation enforced?"

LeIndex: src/http/request_validator.rs    validate_session   (0.92)
         src/middleware/input_guard.rs    authorize_user     (0.84)
         callers: 3 · callees: 7 · blast radius: 12 files

v2.0.0 — the resource revolution, completed

v2.0.0 rebuilt LeIndex around one idea: a code index should never be the biggest process on your machine. The architecture work (one user-scoped daemon + tiny stdio shims, content-addressed generation storage, a streaming bounded indexing pipeline, an admit/defer/reduce memory controller) is benchmarked in BENCHMARKS.md — including MRR@10 = 1.0000 against the v1.9.5 baseline, so none of it cost retrieval quality. The final rounds of hardening, all measured on this repository's own source (457 files / 20,500 nodes / 122,000 edges):

How: a client-side content-addressed embedding cache (probe hits locally, embed only misses — an all-hit run never loads the model), edge-level PDG diffing on save (unchanged graphs write nothing), a single-daemon policy with zombie-aware cleanup and a sibling-aware memory floor, and honest reporting end to end (impact direction labels, impact_note when a zero could mean degradation, signature_scope on incremental counts).

The intelligence stack behind it

• 37 active language grammars today — Rust, TypeScript/JavaScript, Go, Python, Java, C/C++, C#, Ruby, PHP, Swift, Kotlin, Scala, Elixir, Erlang, Haskell, Zig, R, and more — on a roadmap to 100+; each grammar is an individually gated crate that cannot regress the build (docs/LANGUAGES.md).
• Documentation is first-class: markdown/rst/adoc/txt files index as heading-section nodes, so "how do generations work" retrieves the exact ARCHITECTURE section — not the whole handbook.
• Leiden community detection (feature-flagged, default on) clusters the call/data/containment graph; project-map can group by community and impact-analysis reports community crossings.
• SCIP precision tier (on by default; LEINDEX_FEATURE_PRECISION_INGEST=false disables): when a language indexer such as rust-analyzer scip is present, LeIndex merges its exact definitions and relationships into the PDG — upgrading heuristic edges to confidence 1.0 and marking precision-confirmed symbols. Missing indexers degrade silently to the tree-sitter tier.
• Engram query phrase-book (opt-in: LEINDEX_FEATURE_ENGRAM=1): a persistent, content-addressed table of neural query embeddings under ~/.leindex/engram/. A repeated query is served without waking the embedder (no worker spawn, no network round trip); keys include the embedder identity, so a changed model never serves stale vectors. Bounded (20,000 rows / 256 MiB, LRU) and reported in leindex_analyze mode=diagnostics. Query embeddings only; index-time neural reuse is the existing global embed cache.

The retrieval quality claims are gated by a deterministic benchmark — internal fixtures plus a vendored subset of the CoSQA (ACL 2021) human-annotated query/code benchmark — see docs/baselines/AGENT_TASKS_METHODOLOGY.md.

What agents see: 4 MCP tools

Four routers instead of twenty schemas — a mode / action argument picks the operation, every other argument passes straight through:

find is a native trigram index (the technique behind Zoekt) built with each index run: about a millisecond on a 700-file repository, results always read from the live file, and any directory outside the index is scanned on the fly with no setup. Every branch takes tier (l0 card · l1 overview · l2 full detail), and argument detail lives in the leindex://tools/guide MCP resource. The original per-tool names (leindex_search, leindex_edit_apply, …) still work as direct calls.

Every tool is also on the CLI — the same handlers, the same output:

leindex tools list --verbose
leindex tools inspect leindex_explore
leindex tools run leindex_explore --set mode=find --set pattern=retry
leindex tools run leindex_explore --args '{"mode":"search","query":"retry policy","top_k":5}'

90%+ token savings for AI coding tools

Standard tools burn context reading whole files to find one function. LeIndex returns the structured answer:

Each call is context-aware, not atomic: symbol lookups return callers, callees, data dependencies, and impact radius; file summaries return cross-file relationships; renames return a previewed multi-file diff. One LeIndex call replaces chains of Grep → Read → Read → Read.

Methodology: docs/TOOL_SUPREMACY_BENCHMARKS.md.

Quick start

Install (pick one — then run leindex setup to provision the neural path):

# cargo (Rust users)
cargo install leindex
cargo install leindex --features onnx --force   # ensure the ONNX runtime is linked
leindex setup

# npm (Cursor / Claude Code / VS Code users)
npm install -g @leindex/mcp
npm run setup --prefix "$(npm root -g)/@leindex/mcp"

# PyPI (Python users; bootstraps the Rust binary via cargo)
pip install leindex && leindex setup

# prebuilt release bundle
curl -fsSL https://raw.githubusercontent.com/scooter-lacroix/LeIndex/master/install.sh -o install-leindex.sh
bash install-leindex.sh && leindex setup

The setup wizard selects local Qwen3 ONNX, managed EmbeddingGemma2 GGUF, an external embeddings server, or the existing configuration. TF-IDF and the PDG remain available without a neural provider. Non-interactive --neural selects local Qwen3; use --embedding keep to retain an existing route. Managed services (leindex services list|start|stop) own the llama-server process on Linux; auto_start = true brings them up on demand for runtime embedding calls (--check never starts services).

leindex setup --embedding qwen3 --cpu
# Linux managed service: explicit compatible runtime, or verified official download
leindex setup --embedding embeddinggemma2 --download-llama cpu --gpu-layers 0
# Alternative: --llama-server /absolute/path/to/llama-server
leindex setup --embedding remote --embedding-endpoint http://127.0.0.1:9000/v1 \
  --embedding-model your-served-model
leindex setup --check

ONNX --cpu/--gpu amd|nvidia selects a local execution provider; llama.cpp --gpu-layers controls managed offload independently. Official runtime backends are cpu, vulkan, and cuda on supported Linux architectures. Managed setup binds loopback only; use an authenticated external server for nonlocal inference. Remote routes do not require ORT or a local Qwen model. --check makes a bounded embedding request without starting services; start a stopped managed service first. See neural setup for authentication, dimensions and templates.

System One (experimental decision gate)

System One is an experimental typed-decision gate: a small decision model validates tool calls before they run (pre-gate) and classifies search results before they reach your agent (post-gate). It fails open — any provider problem leaves calls and results untouched — and ships off by default.

1. Build with the feature: cargo install leindex --features system-one. Prebuilt bundles and default crates.io builds do not include the gate; leindex setup --check reports build (system-one feature): yes/no.
2. Provision a decision model. leindex setup --system-one writes the [system_one] section and, for the default laya provider, asks which Laya backend to use: external (the fresh-install default) prompts for an existing HTTP endpoint, with optional model and api_key_env values (empty = provider default / no key); managed (explicit choice, Linux) prompts for the llama-server binary and an absolute Laya GGUF path (for example ggml-org/Laya-GGUF on Hugging Face) and writes a [services.laya] section the gate derives its endpoint from. One llama-server serves one model, so the embeddings service must use a different port. On GPUs, pin the device explicitly (e.g. extra_args = ["-ngl", "99", "--device", "Vulkan1"]) — auto device selection can pick an integrated GPU.
3. Arm the runtime flag in every process that serves MCP or runs the CLI: export LEINDEX_FEATURE_SYSTEM_ONE=1 (and set the same variable in your MCP client's env block). Configuring [system_one] alone never enables the gate.
4. Verify: leindex setup --check reports the gate active with probe: provider answered. The probe sends a real decision request (timeout: system_one.timeout_ms). --check never starts the managed service: start it beforehand or the probe fails, even with auto_start = true.

Fail-open tuning lives in [system_one]: mode (off/shadow — decide and log only —/enforce), pre_gate, post_gate, timeout_ms, and the relevant_min/instructions_max/top_n/tau_apply thresholds. See leindex.toml.example and docs/NEURAL_SETUP.md.

Inspect or edit configuration without rerunning setup:

leindex config path
leindex config show
leindex config get search.search_mode
leindex config set search.search_mode '"hybrid"'
leindex config unset indexing.embeddings.remote_endpoint
leindex config validate

Values passed to config set are TOML; strings need the inner double quotes. Store API-key environment variable names, never credentials, in shared config.

Index and search:

leindex index /path/to/project     # ~seconds; re-indexes are delta-priced
leindex search "authentication flow"
leindex analyze "how authorization is enforced"

Connect your agent (MCP stdio):

{
  "mcpServers": {
    "leindex": {
      "command": "npx",
      "args": ["-y", "@leindex/mcp"]
    }
  }
}

Replace npx -y @leindex/mcp with leindex mcp if you installed the binary directly. To expose experimental features to the spawned server, pass them in an env block — an MCP server only sees variables its client sets:

{
  "mcpServers": {
    "leindex": {
      "command": "leindex",
      "args": ["mcp"],
      "env": { "LEINDEX_FEATURE_SYSTEM_ONE": "1" }
    }
  }
}

Zed, Cursor, VS Code, and Claude Code configuration snippets are in docs/MCP.md. Long-running servers self-exit after [mcp] idle_timeout_secs (default 1800) and evict idle engines after engine_max_idle_secs (default 600).

Environment variables: LEINDEX_HOME (storage root, default ~/.leindex), LEINDEX_PORT (HTTP server, default 47500), ORT_DYLIB_PATH (ONNX Runtime override). Memory rails — LEINDEX_WORKER_MAX_RSS_MB (default 10240), LEINDEX_WORKER_MIN_AVAILABLE_MB (default 2048), LEINDEX_REGISTRY_MAX_HEAP_MB (default 1536), the Engram bounds LEINDEX_ENGRAM_MAX_ENTRIES (20000) and LEINDEX_ENGRAM_MAX_MB (256) — plus the escape hatches LEINDEX_CLI_SHUTDOWN_DAEMON, LEINDEX_ALLOW_MULTIPLE_EMBED_DAEMONS, and the LEINDEX_FEATURE_* rollout kills catalogued in .env.example. The full System-One walkthrough — build flag, wizard, runtime flag, MCP env block, verification — is in the setup section. [system_one] provider keys live in leindex.toml.example (laya targets a local Laya model — managed llama-server backend or an external endpoint with an optional api_key_env name; the remote jev/typesafe/openai providers take base_url, model, and api_key_env — the environment variable name, never the key itself).

How it works

Tree-sitter parses every source file into symbols that feed two layers over the same nodes: a TF-IDF lexical corpus and a Program Dependence Graph (call edges, data flow, containment). The configured neural worker embeds the same nodes for semantic scoring, joined after readiness — terminal failure preserves the complete TF-IDF/PDG result.

Codebase → Tree-sitter → PDG + TF-IDF (core, always) → neural join → hybrid ranking

• Indexing is resumable and delta-priced. Phase checkpoints (scan → parse → pdg → lexical → neural) resume after interruption; re-indexes only re-embed changed content; unchanged PDG edges write nothing (save_pdg diffs at the edge level).
• Generations are immutable and content-addressed. Publication uses a staging→promote pattern with blake3-hashed layers; no-op reindexes are byte-identical, and leindex retention --gc keeps the store bounded.
• Memory is bounded by construction. Streaming pipeline, admission control (admit/defer/reduce — never error), a single RSS-capped embed daemon with idle eviction, and byte-budgeted registry eviction.

Use cases

• Understanding unfamiliar codebases — ask questions instead of reading every file
• Onboarding — find relevant code without tribal knowledge
• Legacy exploration — surface logic buried in decades of code
• AI coding assistants — give LLMs real structural context over MCP
• Refactoring with confidence — impact analysis and previews before touching disk
• Cross-project search — query multiple repositories at once

Under the hood

• 18 MCP tools — search, navigation, analysis, and safe editing (preview/dry-run/preview-only defaults on every destructive path)
• Flexible embedding backends — TF-IDF, local ONNX models (Qwen3), a managed llama.cpp GGUF service (EmbeddingGemma2), or remote OpenAI-compatible providers
• System One decision gate (experimental) — pre/post-gate validation of MCP calls and search results by a local typed-decision model, off by default, fail-open
• Honest degradation everywhere — direction labels on impact figures, notes when enrichment was skipped, signature_scope on incremental counts, freshness footers separated from machine-readable JSON
• 5-phase analysis — structural scan, dependency map, logic flow, critical path, synthesis; freshness-aware and incremental
• Dashboard — Bun + React operational UI with project metrics and graph telemetry
• HTTP + WebSocket server — leindex serve
• Flexible embedding backends — TF-IDF, local ONNX models, or remote providers (OpenAI, Cohere)
• Built in Rust — fast, low-memory, safe concurrency

Learn more

• BENCHMARKS.md — resource benchmarks, model bake-off, acceptance-gate evidence
• docs/NEURAL_SETUP.md — EmbeddingGemma2 remote preset, llama-server/vLLM recipes, local CPU/GPU provider setup, and fallback troubleshooting
• docs/MCP.md — MCP integration for every major client
• CHANGELOG.md — full release history, including the v2.0.0 post-release fix rounds
• ARCHITECTURE.md / RUST_ARCHITECTURE.md — system design

License

MIT OR Apache-2.0 — see LICENSE.