Semlith

A local vector store and code graph for AI agents. Index your code and documents once, keep the index current as you save, and answer an agent's questions in milliseconds — without anything leaving the machine.

Install · Quick start · Agents · Portal · Benchmarks · Docs

Why Semlith

An agent working in a large codebase either reads too much, which is slow and expensive, or guesses which file to open, which is often wrong. What it needs is the few spans that answer the question. Semlith finds them.

• Hybrid search. Every query runs against a vector index (meaning) and a keyword index (exact names), and the two rankings are fused. How does retry backoff work and EMBED_BATCH both land on the right chunk.
• A code graph built on the same pass. Definitions become nodes; calls, imports and re-exports become edges, each labelled with how well it is supported. Ask what calls a function, what breaks if it changes, or how two symbols connect.
• Always current. A watcher re-embeds a file when you save it. There is no build step and nothing to go stale against your working tree.
• Local by design. No API keys, no network at query time, a portal bound to 127.0.0.1 only. Model weights are the one download, pinned by digest.
• Made for agents. An MCP server with sixteen tools, registered in your agent clients by one command, and a ledger of every retrieval with the tokens it saved.
• More than code. PDFs, Office documents, notebooks and Markdown are read as text; images are searchable by what they show.

Install

One command, no Rust toolchain needed.

macOS and Linux

curl --proto '=https' --tlsv1.2 -fsSL https://raw.githubusercontent.com/semlith/semlith/main/install.sh | sh

Windows

irm https://raw.githubusercontent.com/semlith/semlith/main/install.ps1 | iex

The installer picks the release for your machine, checks it against the release's SHA256SUMS, unpacks it into ~/.semlith/bin and runs semlith setup. Setup puts semlith on your PATH, downloads the embedding model, registers semlith with the agent clients it finds, and installs the daemon as a login service. Run semlith setup again at any time to repair an install; --yes takes every default.

Other routes: cargo install semlith, or an archive from the releases page. semlith upgrade replaces the binary with the newest release; semlith never checks for updates on its own.

Requirements. A 64-bit machine running macOS on Apple silicon, Linux on x86_64 or aarch64, or Windows on x86_64 (Windows PowerShell 5.1 is enough). The Linux archives ship libonnxruntime.so beside the binary and need only glibc 2.34 or later and libstdc++ (GLIBCXX_3.4.22): Debian 12, Ubuntu 22.04, RHEL 9 and Amazon Linux 2023 all qualify. Intel macOS is not supported, because ONNX Runtime no longer ships for it.

Verifying the installer

The installer writes the binary to ~/.semlith/bin (mode 700 on macOS and Linux) and, on Linux, libonnxruntime.so beside it. Nothing is written until the archive passes two checks: its SHA-256 against SHA256SUMS, and — when gh is on your PATH — its GitHub artifact attestation, signed by this repository's release workflow. Without gh it says provenance was not verified. curl is held to https and TLS 1.2+, and the script runs only once it has fully downloaded. Releases before v0.36.0 carry no attestations.

To read the installer before running it:

curl --proto '=https' --tlsv1.2 -fsSLO https://github.com/semlith/semlith/releases/latest/download/install.sh
less install.sh
gh attestation verify install.sh --repo semlith/semlith
sh install.sh

On Windows: irm https://github.com/semlith/semlith/releases/latest/download/install.ps1 -OutFile install.ps1, read it, then powershell -ExecutionPolicy Bypass -File install.ps1.

To verify an archive by hand:

grep "  semlith-<tag>-<target>.tar.gz$" SHA256SUMS | shasum -a 256 -c
gh attestation verify semlith-<tag>-<target>.tar.gz --repo semlith/semlith

Quick start

cd ~/src/ripgrep
semlith index .                                   # one store; the model (~52 MB) downloads once
semlith search "where is a file detected as binary"
semlith neighbors search_parallel                 # callers and callees
semlith start                                     # the daemon and the portal

$ semlith search "where is a file detected as binary" -k 1
1. 0.083 vf  GUIDE.md:735-746
   binary files:

   1. The default mode is to attempt to remove binary files from a search
      completely. ...

$ semlith neighbors search_parallel
callers (2)
  main via defines (extracted)  crates/core/main.rs:44  · call at line 166
  run via calls (resolved)  crates/core/main.rs:78  · call at line 90
callees (25)
  matcher via calls (resolved)  crates/core/flags/hiargs.rs:379  · called at crates/core/main.rs:177
  search_worker via calls (resolved)  crates/core/flags/hiargs.rs:705  · called at crates/core/main.rs:176
  ...

Every hit is a path:start-end locator you can hand to an editor or an agent; vf says the vector and keyword lists both found it.

Use it from an agent

semlith setup     # registers semlith in every agent client it finds
semlith doctor    # checks each client can actually reach it, and says how to fix it if not

setup registers the MCP server at user scope in every client with a registration command — 12 clients are documented and launched in CI, docs/clients.md has every stanza and the HTTP transport. It also installs the semlith Agent Skill, a steering hook that suggests the semlith call when an agent is about to grep, and a read-only research agent.

semlith mcp serves MCP over stdio; the daemon serves the same tools over HTTP at /mcp, behind an agent key (semlith key show). Eight tools are listed by default, because every definition costs tokens on every request; SEMLITH_MCP_TOOLS=all lists all sixteen.

A bare semlith mcp opens every registered store, so one question can span repositories. Supported MCP versions: 2026-07-28, 2025-11-25, 2025-06-18 and 2024-11-05, each proven by a session in tests/mcp.rs.

The portal

semlith start runs one daemon that owns every store: it holds each store's write lock, watches its folders, re-embeds what you save, and serves a portal at http://127.0.0.1:7365. The URL with its session token is printed once, on stdout. docs/portal.md documents every page.

It is not on the network. The daemon binds 127.0.0.1 only, with no flag to change it. Every page and /api/ route needs the per-run session token, sent in a Semlith-Token request header; without it the answer is 401. The agent key opens /mcp and nothing else. Responses carry a Content-Security-Policy of 'self', no CORS header is sent, and every byte the portal loads is compiled into the binary. --airgap refuses every download that is not already cached. The Privacy page checks these claims against the running daemon; docs/security.md is the full account.

How it works

files ──chunk──> text ──embed──> vectors ──quantize──> index/*.tvim  (turbovec)
                  │
                  └──parse──> symbols, edges ─────────> store.db      (SQLite)

query ──embed──> vector ──search shards──┐
      ──keywords──> FTS5 ────────────────┴──fuse──> ranked spans ──> enclosing symbol

• Reading. Files are read as the text a person would see. Thirteen formats have their own reader (PDF, Word, PowerPoint, Excel, notebooks and more); everything else is UTF-8, cut into line-aligned chunks of up to 800 characters. A Markdown chunk is embedded with its heading path, a code chunk with its enclosing definition.
• Embedding. granite-embedding-small-english-r2 (int8, 384 dimensions, ~52 MB) through ONNX Runtime, fixed per store. The daemon also embeds on the Neural Engine on Apple silicon and on the GPU (Core ML, WebGPU or CUDA), each lane checked against known answers before its first batch. Images go through CLIP ViT-B/32 into a second vector space; matching is by what an image shows, not OCR.
• Storing. index/ holds turbovec shards of 65 536 quantized vectors; store.db holds chunk text, spans, symbols, edges, the ledger and the content hashes that make re-indexing incremental. A save rewrites one shard.
• Graph. All 46 languages carry edges. Each edge is extracted (settled by the file itself), resolved (one candidate), ambiguous (several, and it says how many) or inferred (a bare name match). semlith path refuses to cross a name it cannot pin down unless you pass --all-edges.

docs/architecture.md is the full account.

Commands

semlith <command> --help documents every flag.

Where stores live. ~/.semlith/stores/<name>, listed in ~/.semlith/registry.json, so semlith mcp and semlith start find every store with no flags. A command picks its store from --store / SEMLITH_STORE, then a .semlith beside the corpus, then the registered store covering the current directory, and otherwise creates one.

What gets indexed. Everything under the given paths except .gitignored and hidden files, binaries, files over 8 MiB, archives that expand past 32 MiB, and generated or vendored directories (node_modules always; target, build, dist, vendor beside their manifest — SEMLITH_DEFAULT_IGNORES=0 keeps them). Credentials are never indexed; the grey zone is yours to decide.

The retrieval ledger

Every retrieval — from an agent over MCP, the CLI or the portal — is recorded in its store: the query, the client, the session, the hits and what was read. Each row carries the hash of the row before it, so semlith ledger --verify finds an edited or deleted row. It is local, on by default, and off with --no-ledger or SEMLITH_LEDGER=0.

What it saved on this repository. Ask semlith brief all 107 questions of this repository's retrieval harness against a store of src/, then run semlith ledger --verify: 3 982 tokens per answered question were sent to the agent, against 105 132 for reading the 4.1 files those answers named, whole — 26× less, at 100 % coverage, counted with the store's own tokenizer. It is an upper bound: a file named by two answers is counted twice.

Benchmarks

Public code-retrieval benchmarks, run by bench/scorecard/ on one M1 MacBook Air in October 2026 and published as measured, including where another tool wins. Every table, every chart and the command that reproduces each number are on the scorecard page.

Where semlith does not win: Semble edges it at hit@1 and hit@10 on the 20-instance competitor sample, BM25 beats it on CodeRAG-Bench's RepoEval, and with Grep built in Opus rarely reaches for semlith on its own. The scorecard page has these rows and RepoBench-R.

The numbers

Coverage, read out of the code; tests/readme.rs fails the build if any of these drifts from its source:

Measured on a 4P+4E Apple silicon laptop, with the command that reproduces each; docs/performance.md has the full tables:

Peak memory does not grow with the corpus — plan for about 600 MB whatever you index. Query latency does: the vector scan is linear.

Known limits

• One writer per store. While the daemon or semlith watch holds a store, a second semlith index against it exits and names the holder.
• First indexing is bound by the model: about 230 chunks/s on an M1's Neural Engine, 63 on its CPU and GPU, 30 on the CPU alone. Later runs embed only what changed. On macOS the CPU lane alone runs at about 60 % of a terminal's speed inside the login service, because of how launchd schedules agent threads.
• On a Mac the GPU does not embed beside the Neural Engine unless you run semlith accel on gpu-beside-ane. CUDA runs on Linux x86_64 only; an NVIDIA card on Windows embeds through WebGPU.
• A new binary on macOS asks again for access to folders such as Documents, and a store there waits, showing running, until the prompt is answered.
• A store larger than SEMLITH_INDEX_MEMORY reads shards from disk on every query; raise the budget if the store fits in memory.
• The default model is English-only and fixed when a store is created. Image search is not OCR.
• Search filters are SQLite GLOB (no regex); --lang maps extensions and never reads contents. Multi-store search merges rankings rather than ranking jointly. Cross-encoder rescoring (SEMLITH_RERANK=on) is off by default: it costs about 16× the search time for two more answers in 77 at k=1.
• No code signing, no ARM64 Windows or Intel macOS build, and semlith upgrade only replaces a binary in ~/.semlith/bin.

Documentation

Contributing

Contributions are welcome. CONTRIBUTING.md covers building, the checks CI runs and what is out of scope; AGENTS.md is the same for a coding agent. Everyone follows the Code of Conduct. To report a security problem, see SECURITY.md rather than opening an issue.

Prior art

Semlith's vector index is turbovec by Ryan Codrai (MIT), an implementation of TurboQuant from "TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate" by Amir Zandieh, Majid Daliri, Majid Hadian and Vahab Mirrokni. TurboQuant is data-oblivious: it rotates each vector randomly and quantizes each coordinate on its own, with no training pass. That is what an index refreshed on every file save needs — nothing to gather before the first file, nothing to rebuild as the corpus grows, and a vector added is a vector searchable.

License

Apache-2.0. See LICENSE and NOTICE.

Model weights are downloaded at runtime under their own licenses: the default, ibm-granite/granite-embedding-small-english-r2, is Apache-2.0; the CLIP ViT-B/32 pair, the WebGPU plugin (MIT) and the NVIDIA libraries the CUDA pack fetches carry theirs. See docs/models.md.