scryer-mcp

Model Context Protocol (MCP) server implementation for Scryer.

Responsibilities

• Protocol Engine: Implements the official rmcp v3.5.1 ServerHandler protocol interface.
• 19 Discrete Tools: One tool per question an agent asks, across 7 routers (Administration, Navigation, Dependency Discovery, Symbol Search, Symbol Inspection, Graph Traversal, and Architectural Memory). First-contact tools answer in one call; capped summaries point to deep-dive tools (ADR 0011).
• Tool Annotations: Every tool carries MCP ToolAnnotations (make_tool in src/tools/admin.rs): query tools are readOnlyHint; index_workspace / switch_active_project are idempotent writes; record_adr writes; delete_project / delete_adr are destructiveHint.
• 3-Tier Context Cascade: Resolves project multi-tenancy automatically via canonical path prefix matching, explicit arguments, or session workspace roots.
• Token Budget and Usage Telemetry: Measures exact payload token counts using tiktoken-rs, enforces a per-tool token budget (1,000 by default; see Response Shapes and Budgets) and persists measured usage (tool, response tokens, truncation, latency; never content) to ToolInvocationMetric in Turso. No savings estimate is reported (ADR 0015).
• Shared Daemon (src/daemon/, Unix only): One process owns the database and serves many MCP clients over a Unix socket, each with its own session.
• Architectural Memory: Parses Markdown frontmatter (gray_matter) and document ASTs (pulldown-cmark) into a cached corpus, ranked with BM25, affected_paths globs (globset) and call-graph proximity.

Architecture

Client (AI Agent / IDE) <---> rmcp ServerHandler (ScryerMcpServer)
                                     |
                +--------------------+--------------------+
                |                                         |
     3-Tier Context Cascade                    Token Budget & Telemetry
     (crates/scryer-mcp/src/context.rs)          (crates/scryer-mcp/src/telemetry.rs)
                |                                         |
                +--------------------+--------------------+
                                     |
                             Tool Dispatcher
                                     |
       +-----------------+------------+-------------+------------+-------------+-----------------+
       |                 |            |             |            |             |                 |
Administration     Navigation    Dependencies   Search        Inspection    Graph Traversal  Architectural Memory
(src/tools/admin)  (navigation)  (dependency)   (search)      (inspect)     (src/tools/graph)(adr, adr_rank)
- list_projects    - resolve_def - get_crate_.. - search_sym.. - inspect_sym. - trace_call_... - query_adrs
- switch_active... - find_refs                                              - calculate_...  - record_adr
- delete_project   - get_scope                                                               - delete_adr
- index_workspace  - get_outline
- get_indexing...  - get_type_..
- get_token_...

Tool Contract

How the tools behave when an agent calls them slightly wrong. Each point has a test in tests/tool_contract_test.rs.

• Published schemas are portable (tools/schema.rs, simplify_schema): plain type (no ["string","null"]), inline enum, no $ref / $defs / format / minimum, because Gemini-style function declarations and strict validators reject those. A unit test fails if any schema regains one.
• Closed value sets are enums (tools/enums.rs): scope_level, role, direction and status list their valid values in the schema, and a wrong value is an error naming the choices. direction also accepts callers / callees.
• max_tokens, no_truncate and project are described once in schema.rs, and published only where they matter (BUDGET_PARAM_TOOLS: the five tools with large outputs; PROJECT_PARAM_TOOLS: the five project-admin tools). Publishing and accepting are separate: accepted_tool_definitions() (tools/mod.rs) keeps every parameter and is what coerce_arguments and dispatch use, while all_tool_definitions() applies publish_schema. A call that still sends a hidden parameter works (parameters_that_are_not_published_are_still_accepted_and_coerced).
• Arguments are coerced to the published types (coerce_arguments): "12" for an integer, "true" for a boolean, a bare string for an array, and null for an absent optional all mean what the agent intended. A string that isn't a number is still an error.
• An explicit project must match a registered project's slug, ID or root path (ProjectContextResolver::resolve_project); otherwise the call errors instead of falling back to the active or only project. switch_active_project and index_workspace (nonexistent or non-directory path) fail the same way.
• Required combinations are enforced with a message: calculate_blast_radius needs symbol or file_path; search_symbols exactly one of query / like_symbol; delete_adr one of adr_number / file_path / title; scope_level: "file" needs file_path. Lines are 1-based; line: 0 is an error.
• References name their container: each find_references item carries enclosing_symbol (the qualified name of the function or type it sits in), so "who calls X" is one call, not one get_enclosing_scope per site.
• Empty means explained: find_references, calculate_blast_radius, get_type_contract, get_enclosing_scope, query_adrs and get_file_outline return notes when the name, path or filter matched nothing; resolve_definition puts the reason in notice; get_crate_outline explains in formatted_markdown. An unknown name is distinguishable from "no usages".
• Server instructions (server_instructions(profile) in server.rs, sent at initialize) carry the which-tool decision tree and these conventions for every client. There is one text per profile; a test per profile fails if it names a tool the profile doesn't list or leaves a listed tool unreachable, and both must stay within 1,800 characters (Claude Code silently truncates at 2,048).
• Definitions have a size budget (the_tool_list_stays_small, every_tool_definition_fits_its_budget): about 13.6k characters in total (21.3k before Plan 17), each description at most 600. cargo test -p scryer-mcp --test tool_contract_test -- --ignored --nocapture print_definition_sizes prints the per-tool figures.

Response Shapes and Budgets (Plan 19, ADR 0014)

Agents lose on turns, not accuracy, so answers are built to need no second call and to say how complete they are.

• Compact JSON, no waste: results are serde_json::to_string (no pretty-printing), omit nulls and empty arrays, and never carry byte offsets. response_shape_test.rs fails if any default response has a null or start_byte.
• Budgets fit by structure (tools/budget.rs): list results implement Paged and fit_list_to_budget keeps the largest prefix of items that fits (measured in the chosen format, with every field the cut will add). A cut result sets complete: false, next_offset and a see_also naming the call, so the rest is one call away. telemetry::enforce_limit remains the last resort for free-form output; it now cuts at a line (or, for single-line compact JSON, a comma) and names max_tokens: 2×. Outlines shed detail first (navigation::fit_outline_detail); a scope answer shortens its snippet (fit_scope_to_budget). Budgets with no explicit max_tokens: default_budget(tool, items).
• find_references views: callers (one entry per enclosing function: first usage, its source line, count), files (distinct files with counts), sites (every usage with text); exactly three values in one envelope. total_files, total_callers and name_matched (usages tied by name only) describe the whole answer, not just the page.
• Same-named definitions are grouped, not merged (find_references, calculate_blast_radius, trace_call_hierarchy): ambiguous, definitions/candidates with usages, largest first.
• Text format (format: "text", session default via scryer serve --format text; not published in tool schemas, see schema.rs): find_references, search_symbols, get_file_outline, trace_call_hierarchy and get_crate_outline return a header line and one line per item (tools/render.rs). The header reads tool subject: summary | range | complete / TRUNCATED: call again with offset=N | index: files, parse failures, indexed <time> | CAVEAT ...; other notes follow as note: lines. JSON is the default until the agent A/B (follow-up rate, argument filling) shows text does not regress.
• complete is set on a find_references answer only when RECALL_GATE_PASSED (server.rs; Rust recall at least 98% at 60 identifiers on cargo and this repository), every usage is in a .rs file, the answer is not cut or ambiguous, and no file failed to parse (otherwise a CAVEAT note names the failures). It means exhaustive over the indexed files, not "every usage in the world".
• Measuring it: cargo run --release -p scryer-mcp --example response_bench -- --db <copy of work/<suite>/db/scryer.db> --project <suite> --probes testing-harness/probes/<suite>.json --baseline testing-harness/results/response-bench/<suite>-baseline.json prints chars, tokens, calls and answer hits per probe and exits 1 when an answer is lost or calls rise (docs/learnings/response-bench.md). testing-harness/harness.py metrics gives follow-up and cross-check rates per tool from saved agent transcripts.

Session Options and Tool Profiles (src/session.rs)

SessionOptions { tools, max_tokens, project, always_load, format } is what scryer serve flags (--tools, --max-tokens, --project, --always-load, --format) become. ScryerMcpServer::new keeps default options; use .with_options(..) and then apply_session_options().await (which makes project active, after the cwd is set, and fails on an unknown project).

• Profiles (ToolProfile): full lists all 19 tools; core lists tools::CORE_TOOLS (search_symbols, inspect_symbol, find_references, get_file_outline, get_enclosing_scope, calculate_blast_radius). The profile only changes tools/list, the instructions and see_also (which gains (CLI: scryer <command>) for hidden tools); dispatch_tool accepts every tool name under either profile. See ADR 0012.
• alwaysLoad: with AlwaysLoad::Core (default) the core tools carry _meta["anthropic/alwaysLoad"] = true, so Claude Code loads them without ToolSearch. Other clients ignore it.
• Budget precedence: a call's max_tokens wins over the session's --max-tokens, which wins over the tool's default (tools::budget::default_budget: 1,000; 1,500 for find_references and search_symbols; 2,000 for get_file_outline; 600 for get_enclosing_scope, more for a batch; 500 per name for a batched inspect_symbol). Only the response-size budget uses the session value; the outline docstring rule ("full docs when max_tokens is passed") still keys on the explicit argument.

Index State, First Use and Freshness (src/index_gate.rs, Plan 20)

A tool call never answers from an index that is half built, and sees the caller's own edits. EngineService keeps an IndexState per project (scryer-engine/src/index_state.rs): Ready, Building { done, total } (the first index of a project; shared by every session, the watcher and CLI runs) or Stale { pending } (a re-index over an existing index, listing the files it has not updated). index_project sets it under the per-project write guard and clears it when the run ends, however it ends.

dispatch_tool wraps every tool that reads the index (index_gate::needs_index):

1. Resolve and register. If no project matches and the session's working directory is a project root, it is registered and indexed in the background first.
2. Wait. A call that arrives while the first index runs waits for it, up to SCRYER_FIRST_QUERY_WAIT seconds (default 20; ScryerMcpServer::with_first_query_wait in tests). If the wait runs out, the answer says INDEXING 62%: results so far are partial; retry in about N s, carries no complete, and drops the "no indexed symbol named X" notes, because they would be false.
3. Refresh the files the call names. The watcher's in-flight batches are drained (at most 50 ms), the file_path argument is checked, and after the handler answers, every file the answer names (file_path, location and callers/callees entries) is checked again: EngineService::refresh_files stats it (size and mtime, remembered in memory for files whose hash matched the index; the index stores no mtime) and re-indexes a file that changed, drops one that was deleted, and indexes a new source file that is asked about. At most 64 files per call. If anything changed, the handler runs once more on the refreshed index.
4. Say what happened. The text header's index part reads index: fresh, <facts> or index: refreshed 2 files, <facts>; JSON gets a refreshed 2 files: a.rs, b.rs note. During a re-index (Stale), a call that touches only unchanged files answers normally; one that names a pending file says STALE: k file(s) pending re-index (a.rs, ...): results for them may be out of date; retry in a few seconds.

The watcher turns a burst of more than 32 file events into one parallel index_project instead of file-by-file updates (branch switch, pull, formatter run). Measured timings are in docs/learnings/indexing-freshness.md.

Tool Routers & Inventory

1. Administration Router (crates/scryer-mcp/src/tools/admin.rs)

2. Navigation Router (crates/scryer-mcp/src/tools/navigation.rs)

3. Dependency Discovery Router (crates/scryer-mcp/src/tools/dependency.rs)

3b. Symbol Search Router (crates/scryer-mcp/src/tools/search.rs)

• Results carry score (2 dp), matched_terms, and for dependency hits crate_name / version; total, has_more and offset support pagination (limit defaults to 20, max 50).
• like_symbol builds the query from the target symbol's own name, module path, signature and docstring, and excludes the target from the results.
• scope_level: "dependencies" searches only the packages the project links (project_dependency); crate_name narrows to one crate (case-insensitive) and falls back to any cached version of that crate when the project doesn't link it.
• The index lives in memory in EngineService (see crates/scryer-engine/README.md, "Lexical Search") and is rebuilt after the project is re-indexed.

3c. Symbol Inspection Router (crates/scryer-mcp/src/tools/inspect.rs)

Every section is a capped summary; with all sections on, the default response fits the 1,000-token budget by design (worst-case fixture: ~920 tokens).

• Resolution (dependency::lookup_symbol_candidates): exact name, or qualified name / ::- or .-suffix when symbol is qualified, in the active project, skipping reexport stubs. With no workspace hit it falls back to the project 0 dependency cache (linked crates first); a crate-qualified public path with no exact match (e.g. tokio::sync::Mutex, defined at tokio::sync::mutex::Mutex) falls back to the bare name within that crate. file_path (relative or absolute, suffix-matched) narrows candidates.
• Ambiguity: more than one candidate returns ambiguous: true, candidate_count and up to 10 candidates (qualified_name, kind, location); sections are never merged across candidates.
• Header: qualified_name, kind, file_path (project-relative, or absolute into the Cargo cache for dependencies), start_line/end_line, signature, docstring (first 3 lines / 400 chars, docstring_truncated).
• Snippet: read from disk by the stored line range, capped at snippet_lines (default 25) and, when the caller sets none of snippet_lines / max_tokens / no_truncate, at 1,200 characters; lines over 200 chars are cut with …. When cut: snippet_truncated, snippet_remaining { file_path, start_line, end_line } (first unshown line → symbol end) and a see_also line "rest of body: lines X-Y in <path>". A BLAKE3 mismatch against SourceFile.content_hash adds the note "file changed since index"; an unreadable file drops the snippet with a note, never failing the call.
• Token budget: the payload limiter cuts line by line and the snippet is a single JSON line, so server.rs calls inspect::fit_snippet_to_budget after the handler: if the response exceeds the budget (default 1,000 tokens; max_tokens overrides, no_truncate skips), the snippet is shortened to the largest prefix that fits (same snippet_truncated / snippet_remaining / see_also pointers, plus a note) instead of losing every later field. get_crate_outline does the same through dependency::fit_crate_outline_to_budget, keeping the first modules and types that fit, rebuilding formatted_markdown from them and reporting omitted.
• Callers / callees (functions, workspace only): distinct 1-hop calls / implements / instantiates neighbours as "name — path:line", capped at limit (default 5), with caller_count / callee_count.
• reference_count: navigation::count_references for the resolved definition only (workspace: same-project references; dependency: all projects, reexports edges, and references to workspace use stubs that import it, matched by crate and item name via dependency::imports_symbol).
• Invariants (workspace only): top 3 from adr_rank::rank_invariants_for_symbol as { adr_number, title, reason }.
• Members (struct / enum / trait / type / class / interface): first 8 from EngineService::get_type_contract plus member_count, replacing callers/callees.
• include: any of snippet, callers, callees, references, invariants, members; the header is always returned. Unknown names add a note.
• see_also names the deep-dive tool for every capped section (trace_call_hierarchy, find_references, get_type_contract, query_adrs).

4. Graph Traversal Router (crates/scryer-mcp/src/tools/graph.rs)

5. Architectural Memory Router (crates/scryer-mcp/src/tools/adr.rs)

ADR Ranking (crates/scryer-mcp/src/tools/adr_rank.rs)

• Corpus: every .md file directly under docs/adr, docs/learnings and docs/plans, plus DB-only ArchitecturalDecision rows. Cached per project in a process-wide map (daemon sessions each build a fresh server) keyed on the (max mtime, file count) of those directories; record_adr / delete_adr invalidate it explicitly.
• affected_paths globs (globset, case-insensitive, * does not cross /): a bare entry foo/bar matches foo/bar and foo/bar/**.
• query_adrs: BM25F over title (×3), decision (×2), context / consequences / body / affected_paths (×1), plus +3 when a path-like query (contains / or has an extension) is covered by an ADR glob or contains one (src/net → src/net/client/**).
• Symbol invariants (rank_invariants_for_symbol, surfaced by inspect_symbol): resolves the symbol to its definition and file, then scores each ADR: +5 glob covers the symbol's file, else +3 covers an ancestor directory; +2 covers a 1-hop caller/callee file (calls / implements / instantiates edges; only when the file itself isn't covered); +0.2 per path segment of the matching glob (max +1) so exact-file ADRs beat crate-wide ones; and a BM25 text match on the symbol's name, module path and docstring normalised to 0..3.
• Every match keeps the original fields (relevance_score is the rounded score, at least 1) and adds score and match_reasons, e.g. "affected_path 'src/net/**' covers callee src/net/client.rs", "text: ingest, batch".

ADR Lifecycle and Removal Mechanics

1. Creation (record_adr): Creates a numbered Markdown file in docs/adr/{:04}-{slug}.md with YAML frontmatter, then synchronizes it into the Turso database table ArchitecturalDecision.
2. Deletion via MCP (delete_adr): Deletes the markdown file from disk and purges the corresponding Turso record in one operation. An explicit file_path must resolve to a .md file inside the project's docs/adr directory (absolute paths and .. escapes are refused). Once an ADR number is known, only that ADR's rows are removed; a bare title matching several ADRs with different numbers is an error asking for adr_number.
3. Write order and escaping (record_adr): the DB row is inserted first and the file created exclusively (create_new); if the file write fails the row is deleted again, so no orphan is left. Titles are collapsed to one line, and title, status and affected_paths are written as escaped YAML scalars, so quotes or newlines can't break or forge the frontmatter.
4. External File Deletion (e.g. rm, git checkout): If an ADR file is deleted directly on disk outside MCP, query_adrs performs self-healing reconciliation: it detects the missing file on disk and purges the orphaned record from the Turso database (when the ADR corpus is next rebuilt, which the file deletion itself triggers).

Context Resolution Cascade

For any incoming tool request, Scryer determines the tenant project_id using a 3-tier waterfall:

1. Tier 1 (Canonical Path Matching): If file_path is present, resolve longest prefix match against registered Project.root_path entries.
2. Tier 2 (Explicit Project Parameter): If project is passed, lookup Project.slug or ID in ProjectRegistry.
3. Tier 3 (MCP Client Session Root / Active Project): Fall back to session active project ID (set by switch_active_project) or session roots.

Session roots come from the client's roots/list when it supports roots (refreshed on notifications/roots/list_changed); otherwise from the session cwd sent in the daemon handshake. Relative file_path / path arguments are joined onto the session cwd before resolution. All of this state is per session: ScryerMcpServer::new builds a fresh ProjectContextResolver for each daemon connection and each HTTP session.

Shared Daemon (src/daemon/)

A local Turso database file takes an exclusive lock, so only one process may open it. The daemon is that process; scryer serve (stdio) is a thin client that proxies to it.

Handshake bytes are read through a BufReader that is then handed to rmcp, so MCP messages sent in the same write as the handshake are not lost. Index writes are serialized per project in EngineService, so watchers, tools, and CLI runs never index the same project concurrently.

Token Budget, Usage Telemetry & Payload Controls

All tool calls are intercepted by TokenSavingsMiddleware:

1. Measures exact response payload token count using tiktoken-rs (cl100k_base).
2. Persists the call asynchronously to ToolInvocationMetric in Turso: tool, session, project, response tokens, whether the answer was truncated (cut at the budget, or a page with next.next_offset), latency. Never arguments or answers. SCRYER_TELEMETRY=off disables it.
3. get_token_savings_metrics (and scryer report) aggregate those rows into measured usage per tool: calls, response tokens, truncations and a follow-up rate (a call followed by another Scryer call in the same session; a lower bound, since a grep after an answer is invisible to the server). The earlier whole-file-read savings estimate and dollar figure are retired because agents do not read whole files; cost and accuracy claims come from the harness (testing-harness/).
4. Enforces a per-tool default budget (1,000 tokens, more for the tools above) per payload; see Response Shapes and Budgets.
   • Bypass: Pass no_truncate: true to receive the complete, untruncated output.
   • Custom Budget: Pass max_tokens: N to expand the budget.
   • Selective Querying & Pagination: Collection tools support filtering (query, kinds, start_line/end_line, role, file_filter, compact) and pagination (offset, limit) with structured metadata (total_count, returned_count, offset, limit, has_more).

Usage

use std::sync::Arc;
use scryer_db::{ScryerDb, ProjectRegistry};
use scryer_engine::EngineService;
use scryer_mcp::ScryerMcpServer;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let db = ScryerDb::connect("turso::memory:").await?;
    let engine = EngineService::new(db.clone());
    let registry = Arc::new(ProjectRegistry::new(db.inner()).await?);

    let server = ScryerMcpServer::new(db, engine, registry, None)?;

    // Run over stdio:
    rmcp::serve_server(server, rmcp::transport::io::stdio()).await?;
    Ok(())
}