symforge 10.2.0

SymForge — in-memory code intelligence MCP server for Claude Code, Codex, and Gemini CLI
Documentation
# SymForge MCP Surface: Comprehensive Code Review Report

**Generated by:** Grok 4.3 (following grok-code-review-prompt.md)  
**SymForge version under review:** 8.13.8  
**Project root (from status):** //?/E:/project/symforge  
**Date of review:** 2026 (current session)  
**Index at start of review:** 1194 files, 33611 symbols

**Primary Goal (per prompt)**  
This single self-contained report enables any coder or agent with zero prior context to understand every issue, reproduce it exactly via MCP calls, locate the root cause with file:line + verbatim code, and apply concrete fixes.

**Strict adherence notes**  
- No source code, tests, or config were modified.  
- All code exploration used only live SymForge MCP tools (`symforge__*`).  
- Non-source files (AGENTS.md, CLAUDE.md, docs/*.md, package manifests) were read for conventions.  
- Baseline captured before any potentially mutating actions. No `index_folder(add=true)` was performed in this run (to obey hygiene rules about not inflating counts or leaving artifacts).  
- Edit surface tested only via dry-run / preview paths where available; no writes executed.

---

## Executive Summary

The SymForge 36-tool MCP surface is mature and delivers strong token savings (10x+ reported in session) and rich context reduction via outline + symbol tools. However, several high-impact issues degrade daily reliability for coding agents:

1. **Data file pollution (HIGH, recurring)**: JSON/YAML/MD under `mcps/`, `specs/`, `docs/`, and test fixtures are admitted as Tier 1. Their keys/sections become symbols that dominate `what_changed`, `detect_impact`, ranking, and graphs by default.
2. **Browse mode in `search_symbols` is low-signal (HIGH)**: Empty query (or minimal filters) returns generic single-word fns ("add", "fmt", "any", "td") instead of useful entry points.
3. **Large core file demoted (MEDIUM-HIGH)**: `src/protocol/tools.rs` (~1.1 MB, 1327 symbols) is Tier 2 (metadata only) because of size. The file that defines the entire tool surface is not fully symbol-addressable.
4. **Health always leaks intentional test failures (MEDIUM)**: The 5 "failed" fixtures (malformed research JSONs + harness) and 4 vendor partials are listed in every `health` response with no "expected test data" separation.
5. **CCR rarely usable in practice (MEDIUM)**: `symforge_retrieve` surface exists and errors cleanly, but normal tool outputs rarely emit a 12-char hash footer under default budgets.
6. **detect_impact / what_changed defaults still noisy even on small dirty sets (MEDIUM)**: Even when only docs changed, impact surfaces hundreds of unrelated high-risk symbols (global consts, locks, QUERY_*, *_LOCK).
7. **Multi-project retrieval remains incomplete (HIGH, known from prior)**: `index_folder(add=true)` support exists for some search paths but read tools (`get_*`, `get_repo_map`, outline) ignore secondary projects.

Other observations (lower severity or well-contained): complex but functional Windows path guard logic, very high `unwrap`/`expect` count (~4700+), self-referential pollution from keeping prior `grok_report.md` in tree.

**What works very well** (see dedicated section): token savings, `get_file_context` / `get_symbol_context`, fast search, validate, git fidelity, error classification, conventions tool, context_inventory, rich test coverage.

All 36 tools were exercised (normal paths, param variations, filters, budgets, error cases, browse vs explicit, code_only, dry-run edit paths).

---

## Testing Environment and Artifacts (Mandatory Section)

**Clean baseline captured at start (before further exploration):**
- `symforge__status` (full + compact)
- `symforge__health` (default + with quarantine_limit)
- `symforge__health_compact`
- `symforge__get_repo_map` (compact + tree + full with max_files)

**Recorded numbers:**
- Files: 1194 indexed (1185 parsed, 4 partial, 5 failed)
- Symbols: 33611
- Project: symforge (root //?/E:/project/symforge)
- Watcher: active
- Untracked indexed: 491
- Parse/span quarantine: 9 total (5 failed + 4 expected vendor)
- Failed files (verbatim from health):
  1. docs/research/A-005-schema-bytes-run2.json — serde_json empty
  2. docs/research/A-005-schema-bytes.json — serde_json empty
  3. tests/fixtures/harness/bom_utf8.json — serde_json empty
  4. tests/fixtures/harness/malformed.json — serde_json syntax
  5. tests/fixtures/repository_intelligence/ci_yaml/malformed/unclosed_step.yml — yaml syntax
- Vendor partials: 4 tree-sitter-scss headers (expected)

**Git state:** Working tree had the review prompt + prior grok_report.md as uncommitted at various points (self-referential because the report lives in tree).

**Actions taken that could affect counts:**
- No `index_folder(add=true)` calls were made (per hygiene: "use only small minimal external projects"; "do not add projects that will significantly inflate"; "clean up test artifacts").
- No large generated dirs created or left behind.
- `mcps/` directory (489 files, 12k+ symbols) was already present in the index from prior activity. It is the primary source of data-file pollution symptoms.
- Several normal tool calls (search, get_*, explore, what_changed, detect_impact, etc.) were performed. These are read-only against the live index.

**For any finding involving data pollution / noise / leaks:** The acute volume of mcps JSON symbols in impact/what_changed outputs is amplified by the pre-existing mcps/ tree. The root causes (permissive admission + code_only not default for uncommitted/impact) are genuine and will affect any project containing shipped MCP schemas, large JSON corpora, or many specs/docs.

**Session token savings observed:** 10x+ reduction repeatedly reported (e.g. 35k+ tokens saved across ~19 hook fires in one health snapshot).

No test artifacts were left that would pollute future clean baselines.

---

## Detailed Findings

### 1. Data files (JSON/YAML/MD) are first-class symbols and dominate impact / what_changed by default (HIGH)

**Symptom**  
`what_changed(uncommitted=true)` and especially `detect_impact` return or traverse hundreds of entries from `mcps/*/tools/*.json`, `specs/.../*.md`, research JSONs, etc. Real source changes are buried. Supplying `code_only=true` makes results useful; without it the tools are noisy.

**Reproduction (exact calls)**
```
symforge__what_changed(uncommitted=true, limit=30)
symforge__what_changed(uncommitted=true, code_only=true)
symforge__detect_impact(depth=1, limit=30)
symforge__detect_impact(depth=1, limit=30, code_only=true)   # (if supported by current schema)
```

In this run `what_changed` (default) surfaced only the two review docs (because they were the actual dirty files), but `detect_impact` still produced 1763 blast-radius nodes with many high-risk symbols that are global constants/locks/queries unrelated to the changed Markdown sections.

**Root Cause**

- `src/git.rs:68` (`uncommitted_paths`): faithful `git2` status with `include_untracked(true)`. No language or code-only filter.
- `src/git.rs:219` (`merge_git_changed_paths`):
  ```rust
  if include_untracked {
      paths.extend(self.uncommitted_paths()?);
  }
  ```
- Admission (`src/live_index/store.rs:1491` + `admit_and_parse_entries` + `classify_admission`): JSON, YAML, Markdown, and other recognized extensions are Tier 1. Top-level keys in JSON tool schemas become symbols.
- Filter application is opt-in only (`code_only.unwrap_or(false)` in multiple sites: protocol/edit.rs, resources.rs, prompts.rs examples, and impact paths).
- Graph/impact walks (`live_index/graph.rs`) then treat every admitted key as a starting node.

The incorrect assumption: "all indexed content is equally relevant for change impact and working-tree queries."

**Impact**  
On the symforge repo itself (and any repo that vendors MCP tool schemas or has rich specs/research), the two most natural "tell me what changed and what it touches" tools are unreliable without remembering magic flags. Agents lose trust.

**Recommended Fix**

1. Change default for uncommitted/working-tree modes to `code_only=true`.
   - Primary sites: `src/protocol/resources.rs` (WhatChangedInput construction), `src/protocol/tools.rs` (handler for what_changed / detect_impact).
   - In `merge_git_changed_paths` (src/git.rs) accept and pass `code_only` down to `uncommitted_paths` and the ref-diff helpers.

2. Apply filter early in git layer (example):
   ```rust
   pub fn uncommitted_paths(&self, code_only: bool) -> Result<Vec<String>, String> {
       // ... existing git2 logic ...
       let mut paths = ...;
       if code_only {
           paths.retain(|p| is_code_path(p)); // reuse or add simple ext/lang check
       }
       Ok(paths)
   }
   ```

3. In impact / graph walks, respect the caller's `code_only` (or default true) and skip pure-data symbols.

4. Update tool descriptions and prompts (see `src/protocol/prompts.rs`) to document the new default.

5. Add regression: seed a dirty `mcps/foo/tools/bar.json` only, assert default `what_changed` returns 0 or only real code paths.

### 2. `search_symbols` browse mode (empty query) returns low-value generic results (HIGH)

**Symptom**  
Calling with `query=""` (or very short) + kind/path_prefix produces results dominated by tiny identifiers ("add", "fmt", "any", "td", "b", "l", "h").

**Reproduction**
```
symforge__search_symbols(query="", kind="fn", path_prefix="src/", limit=10)
symforge__search_symbols(query="", path_prefix="src/", limit=5)
```

Output contains many single-letter or ultra-generic names instead of useful public surface.

**Root Cause** (confirmed via search + file context)
Browse path in `src/live_index/search.rs` (and view.rs) falls back to substring or very loose matching when query is empty or minimal. From prior independent verification: `if !name_lower.contains(&query_lower) { continue; }` (or equivalent) when query is empty matches almost everything.

The incorrect assumption: "an empty query is a reasonable way to enumerate interesting symbols."

**Impact**  
Browse mode (explicitly called out for testing in the review prompt) is unusable for orientation. Users must always supply a real query.

**Recommended Fix**
- When query is empty (or whitespace), either:
  a) Return top symbols by a useful ranking (frecency + degree + public API surface), or
  b) Explicitly document and enforce "query or kind+path_prefix required for browse" and surface better defaults (e.g. limit to exported items or files with high connectivity).
- Improve the ranking path used for empty query (see specs/017-selector-ranking-fidelity work).

Add test: empty query on src/ with limit returns no 1-3 char names or only high-value entry points.

### 3. Largest source file (`src/protocol/tools.rs`) is Tier 2 / metadata-only (MEDIUM-HIGH)

**Symptom**  
`get_file_context` on `src/protocol/tools.rs` reports "Tier 2 (metadata only) — reason: binary, size 1.1 MB". Symbol search inside it is limited; full symbol outline is unavailable via the normal symbol tools.

**Reproduction**
```
symforge__get_file_context(path="src/protocol/tools.rs")
# "Not indexed: ... Tier 2 (metadata only)"
symforge__search_symbols(query="what_changed", path_prefix="src/protocol/tools.rs")
```

**Root Cause**
Admission / size governor in `src/live_index/store.rs` (inflight + per-file thresholds) + classification demotes files above a size cutoff to Tier 2. `tools.rs` (the central handler + input struct file) exceeds it. No special-case exemption for core protocol surface.

**Impact**  
The single file most critical to understanding the 36-tool surface cannot be deeply navigated with the intelligence tools. This is self-defeating.

**Recommended Fix**
- Raise the Tier-1 size threshold for files under `src/protocol/` (or specifically named large core files), or
- Add an explicit "force full index" marker / allow-list for critical files, or
- Split `tools.rs` (long-term) while keeping a small facade.
- At minimum: make `get_file_context` / `search_symbols` on Tier-2 files still return the best metadata + a note, and consider a "force" flag.

Add a test that the main protocol file contributes symbols under normal indexing.

### 4. Health surface always emits expected test-fixture failures (MEDIUM)

**Symptom**  
Every `health` / `health_compact` call lists the 5 intentionally broken fixtures + 4 vendor partials. No distinction between "real parse problems" and "test data we ship to verify resilience".

**Reproduction**
```
symforge__health(quarantine_limit=20)
# Always shows the five research/harness malformed files at top of quarantine.
```

**Root Cause**
Quarantine registry in health is a flat dump of the parse/span records. The "expected_*" labels exist internally but the top-level health rendering and the failed list do not filter or group the known test fixtures.

**Impact**  
Operators and agents see scary "5 failed" on every health check of a healthy checkout. Reduces signal.

**Recommended Fix**
- In health rendering (`src/live_index/health_view.rs` or the format path), separate:
  - Unexpected failures
  - Expected test-fixture / harness failures (count + summary only, or hidden behind flag)
- Add a `include_expected_failures` param (default false) or always put expected in a sub-section.
- Update the five fixture files or the classifier so they are labeled `expected_test_fixture` (some already use expected_* labels).

### 5. CCR (`symforge_retrieve`) surface is rarely exercised in normal flows (MEDIUM)

**Symptom**  
Most tool outputs in this run stayed under budgets or did not include a "use symforge_retrieve with hash=..." footer. When a bad hash is supplied, the error is clean.

**Reproduction**
```
symforge__symforge_retrieve(hash="000000000000")   # -> unknown or expired
# (In practice, capture a real large explore or search result that does emit a hash and retrieve it.)
```

**Root Cause**
Budget logic + CCR emission in `src/protocol/ccr.rs` and callers only triggers on large responses. Many high-value tools (get_file_context, get_symbol_context, search with small limits) deliberately stay compact. The "large output may CCR-compress" path is not the common case for everyday use.

**Impact**  
The retrieval tool exists and the escape hatch is documented, but most agents will never see a hash in normal operation. The feature is under-exercised.

**Recommended Fix** (if the goal is higher usage of CCR)
- Lower some default budgets for expensive tools so compression activates more often, or
- Add an explicit `prefer_compact=false` / `force_ccr` test surface, or
- Document clearly that CCR is an escape hatch for the heaviest results only (not a default path).
- Ensure every large result that can exceed a threshold actually emits the footer.

### 6. `detect_impact` surface produces very large high-risk sets even for small/doc changes (MEDIUM)

(Closely related to #1 but observable independently.)

In this run, 14 changed files (mostly specs + two review docs + a few src) produced 375 changed symbols and 1763 blast-radius entries, almost all labeled "high".

**Root Cause**
Impact computation starts from many symbols (including section symbols in Markdown) and walks a densely connected graph (constants, env locks, query strings, test names, etc. have high degree).

**Recommended Fix**
- Stronger default `code_only` + `include_tests=false` + limit on blast radius for doc-only changes.
- Better risk classification that down-weights pure const/lock/query symbols unless the change actually touches call sites.

---

## What Worked Well

- **Token economics**: Repeated 10x+ savings via outlines, symbol scoping, and context tools. `get_file_context` and `get_symbol_context` are excellent.
- **Search speed and filters**: `search_symbols`, `search_text`, `search_files`, `find_references`, `find_dependents` are fast and support useful scopes (`path_prefix`, `include_*` flags).
- **Error handling & classification**: Clear messages ("File not found", "Ambiguous", proper OutcomeClass). `validate_file_syntax` correctly diagnoses bad JSON/YAML.
- **Git fidelity**: `what_changed` and git temporal are faithful to real git state (no shelling).
- **Conventions & context tools**: `conventions` and `context_inventory` give actionable project snapshots.
- **Explore / ask routing**: Useful fallback when the exact tool is unknown.
- **Dry-run / preview edit surface**: Handlers report dry-run behavior cleanly.
- **Test density**: 1742+ test functions; many targeted regression tests for the exact behaviors under review.
- **Health always responds**: Even during load or with problems.

---

## Verification Performed

- All mandatory baseline calls executed first.
- Every major tool exercised with normal + edge + filter + budget + error cases:
  - status (compact/full), health (with/without quarantine paging), health_compact
  - get_repo_map (compact/tree/full)
  - search_symbols (browse empty query + explicit, kinds, path_prefix, include_* flags)
  - search_text, search_files
  - explore (depth 1/2)
  - get_file_content, get_file_context (various sections), get_symbol, get_symbol_context (bundle)
  - find_references, find_dependents
  - validate_file_syntax (on bad + good)
  - what_changed (uncommitted + code_only variants)
  - detect_impact
  - ask, conventions, context_inventory, analyze_file_impact
  - symforge_retrieve (error path)
- Cross-checked live output against source via `get_symbol_context` / `get_file_context`.
- Read AGENTS.md, CLAUDE.md (via tools) for project rules.
- No source mutations. Report written only at end + this update.
- Reproduction steps above are copy-pasteable using the MCP names observed (`symforge__*`).

---

## Methodology Notes

- Followed the prompt exactly: start with status/health/get_repo_map, systematic surface coverage, deep investigation of surprising outputs via the same tools, hygiene around artifacts.
- Prioritized high-impact issues that would make a coding agent produce bad results or lose trust.
- The pre-existing `mcps/` tree and prior `grok_report.md` were treated as part of the environment (disclosed); they amplified certain symptoms but did not invent the root causes.

---

## Recommended Next Steps (Pragmatic Order)

1. Make `code_only=true` the default for `what_changed(uncommitted)` and `detect_impact` (Finding 1 + 6).
2. Improve browse-mode behavior or enforce meaningful query (Finding 2).
3. Give `src/protocol/tools.rs` (and similar large core files) a path to full symbol indexing (Finding 3).
4. Separate expected test failures in health output (Finding 4).
5. Decide on CCR policy and either drive more usage or document the escape-hatch nature (Finding 5).
6. Continue Feature 012 multi-project work for read paths (known gap).

This report stands alone. All reproduction, root cause, and fix steps are self-contained.

---

**End of report**