SIFS

Fast Rust code search for agents.

SIFS indexes repositories in 119.9ms on average, answers normal warm searches in 0.442ms, and reaches NDCG@10=0.8426 on the current annotated 63-repository benchmark corpus. It runs as a CLI, a Rust crate, or a local MCP server for agent clients. Hybrid search combines semantic retrieval with BM25; BM25 mode runs fully offline without loading model files or touching the network.

Quickstart | Features | Install | MCP server | CLI | Rust library | How it works | Benchmarks

Quickstart

Install the public sifs binary, then search a project.

cargo install --locked sifs
sifs search "authentication flow" /path/to/project
sifs search "parse JWT claims" /path/to/project --mode bm25 --offline -k 10
sifs find-related src/auth/session.rs 42 /path/to/project -k 8

The default path is the current directory, and the default mode is hybrid.

sifs search "where is the login redirect handled"

Features

• Search local code with natural-language queries, identifiers, or mixed architecture questions.
• Choose hybrid, semantic, or bm25 ranking per query.
• Run BM25 search offline without tokenizers, safetensors, or Hugging Face model files.
• Use the same engine from the CLI, Rust, or an MCP client.
• Index local directories or shallow-cloned Git repositories.
• Generate agent files and run benchmark diagnostics for quality and latency checks.

Install

Install from crates.io.

cargo install --locked sifs

Install with Homebrew from the public tap.

brew install tristanmanchester/tap/sifs

Build from source when you are working inside this repository.

cargo build --release
target/release/sifs search "authentication flow" .

The sifs-benchmark and sifs-embed binaries are diagnostics. Build them with the explicit diagnostics feature.

cargo build --release --features diagnostics --bins

Run the test suite after changing indexing, chunking, ranking, model loading, or MCP behavior.

cargo test

MCP server

SIFS can install itself as a local stdio MCP server for Codex and Claude Code. Install the binary once, start the shared daemon, then configure agent clients.

sifs daemon install-agent
sifs mcp install --client all

This installs a reusable MCP server instead of pinning the config to one repository. Agent clients can ask SIFS to search the current project, and tool calls can pass repo when they need a specific local checkout or Git URL.

You can still pin an MCP server to one source when you want that behavior.

sifs mcp install --client all --source /path/to/project
sifs mcp install --client codex --source /path/to/project
sifs mcp install --client claude --scope local --source /path/to/project

You can also start the server directly. Without a path it uses the server process working directory as the default source. Passing a path pre-indexes that source, so MCP clients can call search and find_related without sending a repo argument on every tool call.

sifs mcp
sifs mcp /path/to/project

The installer uses the client CLIs when available.

codex mcp add sifs -- /absolute/path/to/sifs mcp
claude mcp add-json sifs '{"type":"stdio","command":"/absolute/path/to/sifs","args":["mcp"],"env":{}}' --scope local

If a client CLI is not available, sifs mcp install --dry-run prints fallback config.

Codex uses ~/.codex/config.toml:

[mcp_servers.sifs]
command = "/absolute/path/to/sifs"
args = ["mcp"]
startup_timeout_sec = 20
tool_timeout_sec = 60

Claude Code project config uses .mcp.json:

{
  "mcpServers": {
    "sifs": {
      "type": "stdio",
      "command": "/absolute/path/to/sifs",
      "args": ["mcp"],
      "env": {}
    }
  }
}

This is a local process with read access to local paths provided in tool calls. Only check project-scoped Claude Code .mcp.json into repositories you trust.

The daemon can also be run manually for debugging.

sifs daemon run --replace-existing-socket
sifs daemon ping
sifs daemon status --json

CLI

Use sifs search for direct search. Use sifs find-related when you already have a file and line and want similar code elsewhere in the same index.

# Search the current directory
sifs search "where is authentication handled"

# Search a local project with hybrid ranking
sifs search "parse oauth callback" /path/to/project --mode hybrid -k 10

# Use model-free offline BM25 search
sifs search "SessionToken" /path/to/project --mode bm25 --offline -k 10

# Search a remote Git repository
sifs search "stream upload backpressure" https://github.com/owner/project

# Find code related to a known location
sifs find-related src/auth/session.rs 42 /path/to/project -k 8

Use --json, --jsonl, or --format for structured output. Use --language, --path, and --context-lines when an agent needs narrower results.

Persistent index caches live in platform cache directories by default: ~/Library/Caches/sifs on macOS and ${XDG_CACHE_HOME:-~/.cache}/sifs on Linux. Use --cache-dir for another cache root, --no-cache to disable persistent caches, or --project-cache to opt into a repository-local .sifs/ cache.

See command-line usage for every command and flag.

Rust library

The crate exposes the same engine used by the CLI and MCP server.

use sifs::{SearchMode, SearchOptions, SifsIndex};

fn main() -> anyhow::Result<()> {
    let index = SifsIndex::from_path("/path/to/project")?;
    let results = index.search_with(
        "where is authentication handled",
        &SearchOptions::new(5).with_mode(SearchMode::Hybrid),
    )?;

    for result in results {
        println!("{} {}", result.chunk.location(), result.score);
    }

    Ok(())
}

Use SifsIndex::from_path_sparse for BM25-only indexes that never initialize semantic state. Use SifsIndex::from_git to clone and index a remote repository. See Rust library usage for model policy, filters, custom extensions, and chunk-level construction.

How it works

SIFS walks repositories with .gitignore-aware file selection, splits files into useful code chunks, builds a sparse BM25 index, and keeps semantic state lazy until a semantic or hybrid query needs it.

Search modes:

• bm25: sparse lexical search for identifiers, symbols, and exact terms.
• semantic: embedding similarity with the configured encoder.
• hybrid: semantic and BM25 rankings fused together, then reranked.

The default semantic encoder is minishlab/potion-code-16M through a local Model2Vec loader. The loader reads model tensors and tokenizer files directly, so the query path stays inside the Rust process after the model is available locally.

Hybrid search over-fetches candidates, normalizes each ranking with reciprocal rank fusion, applies query-aware boosts, and reranks the top results. Symbol-like queries lean more heavily on BM25. Natural-language questions keep more semantic weight.

Use sifs model pull or sifs model fetch to prefetch the default model. Use sifs doctor to check whether semantic search is ready for offline use.

Benchmarks

The current full-corpus benchmark uses 63 pinned open-source repositories, 19 languages, and 1,251 annotated search tasks. SIFS reaches NDCG@10=0.8426, indexes cold in 119.9ms, and answers normal warm queries in 0.442ms.

The comparison table is regenerated from benchmark JSON by benchmarks/plot_sifs_comparison.py. The Semble row is included as a direct comparison to the Python tool; the SIFS headline avoids using that tool's name for the corpus itself.

SIFS reports three timing fields so repeated-query caching is not mixed up with normal search speed:

cold_index_ms
warm_uncached_query_ms
warm_cached_repeat_query_ms

Use warm_uncached_query_ms when comparing normal searches after an index exists. Use warm_cached_repeat_query_ms only for repeated identical queries inside the same process. The older 0.0017ms figure was cached repeat latency, not general warm-query latency.

SIFS also measures how quickly relevant files enter an agent's context. The curve below counts annotated relevant target files as retrieved chunks are added to the prompt budget.

SIFS is strongest on symbol-heavy queries while still doing well on semantic and architecture questions.

The benchmark artifacts live in benchmarks/results. The full methodology, per-language breakdown, extra figures, and React large-repository smoke result are in docs/benchmark-report.md.

Documentation

• Command-line usage: search, find-related, init, model commands, cache commands, and MCP server startup.
• Rust library usage: SifsIndex, search modes, filters, and indexing options.
• MCP server usage: stdio protocol behavior and tool schemas.
• Agent-native scorecard: the agent-facing contract and readiness evidence.
• Benchmarking: quality, latency, embedding, and local smoke benchmarks.
• Architecture: file selection, chunking, embedding, sparse search, dense search, and hybrid ranking.

File coverage

By default, SIFS indexes code files and skips common generated, dependency, and cache directories. It uses the ignore crate, so nested .gitignore files, Git excludes, global Git ignores, and hidden files behave like familiar developer search tools. Text-like documents such as Markdown, YAML, TOML, and JSON are available through library options.

The file walker currently recognizes Python, JavaScript, TypeScript, Go, Rust, Java, Kotlin, Ruby, PHP, C, C++, C#, Swift, Scala, Elixir, Dart, Lua, SQL, Bash, Zig, Haskell, Markdown, YAML, TOML, and JSON extensions.