# LeanKG PRD - Consolidated Tracking Document
**Version:** 3.7.10-ui-v2-load-more
**Date:** 2026-07-21
**Status:** Active Development — **single source of truth** for product requirements + HLD
**Author:** Product Owner
**Target Users:** Software developers using AI coding tools (Cursor, OpenCode, Claude Code, Gemini CLI, etc.)
**Codebase Version:** 0.19.1 (`origin/main`)
> **Task lists + status live in one place (humans + AI agents):**
> - Markdown: [`docs/prd-task-tracker.md`](prd-task-tracker.md) — **all** US / FR / Release tasks + status (**sorted status-first, then Focus P0→P3**)
> - Machine: [`docs/prd-task-tracker.json`](prd-task-tracker.json)
>
> **P0 CLOSED (v3.7.9):** **Procedural ontology auto-update while using** — `US-ONT-PROC-01` / `FR-ONT-PROC-01..03` / `REL-059` **DONE**. Evidence: [`docs/reports/ontology-proc-auto-smoke-2026-07-21.md`](reports/ontology-proc-auto-smoke-2026-07-21.md). YAML watch + boot marker (concepts **and** workflows) + post-index sync + MCP `ontology_control`.
>
> **CURRENT next (P1 — company adoption / cost):** Graphify packaging queue in §1.1 (three-verb → always-on install → honest edges → …). Evidence: [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md).
>
> **Prior P0 mega-graph serve CLOSED** — `US-MG-TOOL-01` / `REL-055` / `FR-SEM-07` / `REL-054` DONE.
>
> This PRD is the SoT for *mission, narrative ACs, HLD, NFRs, glossary*.
> The tracker is the SoT for *task inventory and Done/Pending/Partial status*.
> Do **not** reintroduce status tables or FR checkboxes here — link the tracker instead.
> All prior PRD/HLD files under `docs/requirement/`, `docs/design/hld-leankg.md`, and duplicate `.docs/` PRDs have been merged here. Do not recreate split PRDs — update this file only.
---
## Changelog
### v3.7.10-ui-v2-load-more - Expand pagination + folder sidebar (2026-07-21)
| ID | Priority | Focus | Summary |
|----|----------|-------|---------|
| US-UI2-11 / FR-UI2-13 / REL-061 | Must Have | **P1** | Default expand page 500; **Load more (+200)** merges into graph; `hasMore` fixed |
| US-UI2-12 / FR-UI2-14 | Must Have | **P1** | Hierarchical Folders & files sidebar + session tree across Overview |
**New content:** §3.17 US-UI2-11..12; §5.19 FR-UI2-13..14 / REL-060..061. RCA: `docs/reports/root_cause_expand_examples_hides_src.md`. Deep test: `docs/reports/ui-v2-sidebar-nav-loadmore-deep-test-2026-07-21.md`.
### v3.7.9-ont-proc-auto - Procedural ontology auto-update is P0 (2026-07-21)
> **Trigger:** Live audit — procedural ontology **works** (`kg_trace_workflow`, 10 workflows / 48 steps) but is **static while using**: no watcher, no MCP write path, boot sync only (marker keyed to `concepts.yaml`). Agents editing `workflows.yaml` or reindexing code do not refresh procedural traces without manual `leankg ontology sync` / container restart.
>
> **Product intent:** Make procedural ontology a **live** company capability: debounce re-sync on YAML change during `mcp-http` / `serve`, fix boot freshness for `workflows.yaml`, optional sync after index / MCP `ontology_control`. Do **not** expand P0 to full LLM auto-extraction of workflows (that remains a later Could Have).
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-ONT-PROC-01 | Must Have | **P0** | Procedural ontology stays fresh while LeanKG is in use (no manual sync for YAML edits) |
| FR-ONT-PROC-01 | Must Have | **P0** | Watch `ontology/workflows.yaml` (+ concepts) during MCP/serve; debounce idempotent sync |
| FR-ONT-PROC-02 | Must Have | **P0** | Boot marker / skip logic considers `workflows.yaml` mtime (not only `concepts.yaml`) |
| FR-ONT-PROC-03 | Must Have | **P0** | Hook: after successful index (and optional MCP `ontology_sync` / `ontology_control`) refresh procedural nodes + code_refs |
| REL-059 | Must Have | **P0** | Live smoke: edit YAML → `kg_trace_workflow` updates without process restart |
| FR-A02 | Should Have | **P1** | Remains docs/automation follow-up; P0 implements the runtime auto-update |
**New content:** §3.18 US-ONT-PROC; §5.21 FR-ONT-PROC + REL-059. Demotes company-adoption queue to **P1 next**.
### v3.7.8-ui-v2-service-expand - Service/Folder replace-graph + CodePanel file gate (2026-07-21)
> **Trigger:** UI v2 single-click on Service/Folder called `GET /api/file` with a directory `filePath` → HTTP 400. Multi-service topology had no drill-in to **replace** the canvas. Follow-ups: Sigma callback refs; Render bake; cross-mount `/api/file`; `LEANKG_SERVE_PROJECT` + atomic switch.
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-UI2-03 (tighten) | Must Have | **P1** | `/api/file` only for content-bearing nodes; not Service/Folder/Directory |
| US-UI2-10 / FR-UI2-12 / REL-060 | Must Have | **P1** | Double-click Service/Folder → expand-service **replaces** graph |
**New content:** §3.17 US-UI2-10; §5.19 FR-UI2-12 + REL-060. RCA: `docs/reports/root_cause_api_file_service_folder_400.md`.
### v3.7.8-graphify-ui - Company ROI + Graphify packaging backlog (2026-07-21)
> **Trigger:** Deep compare of local Graphify v0.9.20 vs LeanKG (MCP + ui-v2). Goal: prove LeanKG is the better **company** choice for **AI agent cost + efficiency**, then close packaging gaps Graphify still wins on (always-on install, honest edges, report/HTML artifacts, NL UI).
>
> **Evidence:** [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md) (supersedes Jul-13 matrix for agent/UI gaps that are now closed in MCP).
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-COST-01 / FR-COST-01 / REL-058 | Must Have | **P1** | Manager-facing ROI brief: LeanKG vs grep/cat + vs Graphify (tokens, multi-repo, ops) |
| US-GF-14 / FR-GF-22 | Must Have | **P1** | Three-verb narrative: path · explain · query first in README / AGENTS / skills |
| US-GF-17 / FR-GF-24 | Must Have | **P1** | Always-on graph-first install/hooks (Cursor/Claude/Codex) — primary **cost lever** |
| US-GF-04 / FR-GF-07..09 / REL-043 | Must Have | **P1** | Honest edges (EXTRACTED/INFERRED/AMBIGUOUS) in MCP + ui-v2 |
| US-GF-06 / FR-GF-13 | Must Have | **P1** | Auto `.leankg/GRAPH_REPORT.md` on index + Overview link |
| US-GF-13 / FR-GF-21 | Must Have | **P1** | Bounded single-file HTML export |
| US-UI2-06 / FR-UI2-08 | Must Have | **P1** | Query FAB NL mode → `query_graph` |
| US-UI2-07 / FR-UI2-09 / REL-057 | Must Have | **P1** | ui-v2 production cutover into embed/Docker serve |
| US-GF-15..16 / US-UI2-08..09 / FR-GF-23 / FR-UI2-10..11 | Should Have | **P2** | Install matrix breadth, reflect skill, cluster legend, ops panels |
**Won't Do (confirmed):** multimodal PDF/image/video; NetworkX primary store; 36-lang race; replace Sigma with vis.js-only UI.
**New content:** §1.1 Enterprise ROI; §3.10 US-GF-13..17; §3.17 US-UI2-06..09; §5.9 FR-GF-21..24; §5.19 FR-UI2-08..11; §5.20 cost/ROI FRs.
### v3.7.7-ui-v2 - GitNexus-shell 2D UI rebuild (2026-07-20)
> **Trigger:** Rebuild LeanKG web explorer UX to match GitNexus `gitnexus-web` exploring shell (3-pane layout, Force/Tree/Circles, mega-graph skip) while keeping LeanKG `leankg serve` REST and CodeElement schema. Existing `ui/` + Track E 3D `graph-ui/` remain separate.
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-UI2-01..05 / FR-UI2-* | Must Have | **P1** | New `ui-v2/` graph explorer shell; Vitest+Playwright parity proof |
| REL-056 | Must Have | **P1** | Parity report: shell behaviors vs GitNexus (no agent/analyze in Phase 1) |
**New content:** Section **3.17** UI v2 stories; Section **5.19** UI v2 FRs. ERD: [`docs/erd/ui-v2-erd.md`](erd/ui-v2-erd.md).
### v3.7.6-mega-concept-query - Mega-safe concept_search / query_graph / clusters (2026-07-20)
> **Trigger:** Post-#87 Docker full-tool suite on `ce03fd8` ([`docs/reports/ce03fd8-docker-mcp-full-tool-test-2026-07-20.md`](reports/ce03fd8-docker-mcp-full-tool-test-2026-07-20.md)). Mega HNSW **PASS**. Mega `concept_search` disconnect/timeout; `query_graph` timeout (`all_elements`/`all_relationships`); `get_clusters` intentional refuse. RCA: [`docs/reports/root_cause_mega_concept_query_clusters_2026-07-20.md`](reports/root_cause_mega_concept_query_clusters_2026-07-20.md).
>
> **Product intent:** Prefer-order and NL tools must work on mega mounts via **keyed / frontier-local** queries (FR-SEM-07 pattern). Do not refuse `concept_search`/`query_graph` forever; do not raise RAM as the fix. Mega `get_clusters` serves precomputed `cluster_id` (no live Louvain).
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-MG-TOOL-01 / FR-ONT-MEGA-01 | Must Have | **P0** | Keyed `concept_search` code_ref + typed name fallback |
| FR-GF-MEGA-01 | Must Have | **P0** | Keyed `resolve_to_qualified` + frontier-local `query_graph` BFS |
| FR-CL-MEGA-01 | Must Have | **P1** | Mega `get_clusters` reads precomputed clusters from DB |
| REL-055 | Must Have | **P0** | Live mega smoke: concept_search + query_graph + get_clusters |
### v3.7.5-mega-sem-oom - Mega-graph HNSW semantic_search OOM is P0 (2026-07-20)
> **Trigger:** Post-merge Docker MCP validation on `main` @ `a89a2cc` ([`docs/reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md`](reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md)). Find/lookup, ontology, flow, and `embed_control` idle resume **PASS** on `/workspace-other` (~641k elements, ~147k vectors). **`semantic_search` on mega FAIL:** RSS climbs ~3.3 GiB → `OOMKilled` / HTTP disconnect; logs show deprecated `all_elements()` + skip elements_cache. Same tool on `/workspace` **PASS** (`method: hnsw+rerank`).
>
> **Product intent:** Mega-graph agents must be able to run HNSW `semantic_search` / `kg_semantic_context` under documented MCP memory budgets **without** killing the HTTP server. Prefer-order (`concept_search` → `search_code`) is a temporary agent workaround, not the product fix.
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-SEM-06 / FR-SEM-07 | Must Have | **P0** | Mega-safe HNSW path: no unbounded `all_elements()`; ANN + paginated element hydration only; MCP stays healthy |
| REL-054 | Must Have | **P0** | Live mega smoke: `semantic_search` + `kg_semantic_context` on `/workspace-other` without OOM/restart |
**New content:** Section **3.14** US-SEM-06; Section **5.15** FR-SEM-07 + REL-054; Section **5.17** cross-link. Tracker Focus **P0** open queue = mega-sem OOM (embed-resume / VE remain DONE).
### v3.7.4-mcp-surface - MCP tool surface rationalization (2026-07-18)
> **Trigger:** Overlap review of loosely-overlapping MCP tools (search triple, semantic triple, identity triple, legacy `mcp_*`). Live schemas + handlers contradict some popular characterizations (notably `semantic_search` is dual-path, not ANN-only; `search_code` is ontology-first on mega-graphs; `wake_up` is L0+L1 cached text, not L0 alone).
>
> **Product intent:** Shrink agent confusion **without** losing capability. Highest ROI is honest schemas + prefer-order. Then delete true subsets. Soft-deprecate only where a clear replacement exists. Do **not** hide ops bootstrap tools.
**Decision table:**
| Proposal | Verdict | Product action |
|----------|---------|----------------|
| Document search triple + semantic triple | Agree | Must Have — schema one-liners + prefer-order |
| Delete `mcp_hello`, `mcp_impact`, `get_doc_for_file` | Agree | Must Have — hard remove after matrix update |
| Soft-deprecate `wake_up` | Partial | Should Have — point to `get_overview_context` (**not** `load_layer(L0)` alone) |
| Soft-deprecate `search_by_environment` | Agree | Should Have — most tools already take `env=` |
| Merge `get_doc_tree` + `get_doc_structure` | Partial | Could Have — after mega-graph safety; both currently `all_elements()` |
| Deprecate all `mcp_*` bootstrap (`mcp_status`/`init`/`index`/`install`) | Disagree | Won't Do this release — `mcp_status` is load-bearing |
| Quote surface 64 → ~57 | Fact error | Recount from `ToolRegistry::list_tools()` (≈85 today); near-term shrink ≈85 → ~82 after 3 deletes |
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-SURF-01 / FR-SURF-01 / FR-SURF-02 | Must Have | **P1** | Fix `semantic_search` dual-path docstring; prefer-order on search + semantic tools |
| US-SURF-02 / FR-SURF-03 | Must Have | **P1** | Delete `mcp_hello`, `mcp_impact`, `get_doc_for_file`; update `tests/redundant_tools_matrix.rs` |
| US-SURF-03 / FR-SURF-04 | Should Have | P2 | Soft-deprecate `wake_up` → `get_overview_context` |
| US-SURF-04 / FR-SURF-05 | Should Have | P2 | Soft-deprecate `search_by_environment` |
| US-SURF-05 / FR-SURF-06 | Could Have | P3 | Optional merge / mega-safe `get_doc_tree` + `get_doc_structure` |
| REL-053 | Should Have | P2 | Release note after hard deletes land |
**New content:** Section **3.16** US-SURF-01..05; Section **5.18** FR-SURF-01..06 + REL-053.
### v3.7.3-sem-filter-ops - Live MCP verification + ops fixes (2026-07-18)
> **Evidence:**
> - [`docs/semantic-search-mcp-verification-2026-07-18.md`](semantic-search-mcp-verification-2026-07-18.md) — Docker MCP `project=/workspace`, **3,271** vectors, semantic/kg tools GREEN; **no regressions** from embed-resume. PARTIAL: Probe G (minified `src/embed/assets/*.js`) + Probe H (`src/benchmark/*` verdict noise).
> - [`docs/reports/embed-3-workspaces-2026-07-17.md`](reports/embed-3-workspaces-2026-07-17.md) — cold embed + live vector counts (`/workspace` 3,271; large side mount **146,977**; `/workspace-freepeak` 14,110); semantic_search OK on all three after MCP `mem_limit: 6g` / `mem_reservation: 3g` / `cpus: "6"` + `LEANKG_AUTO_INDEX=0`.
**Product actions this revision:**
| ID | Priority | Focus | Intent |
|----|----------|-------|--------|
| US-SEM-05 / FR-SEM-06 | Must Have | **P1** | Drop `embed/assets/` always; gate `src/benchmark/` unless query says "benchmark" |
| FR-SEM-04 / REL-051 | Should Have | P2 → **DONE** | Live semantic smoke executed 2026-07-18 (complement to cargo suite) |
| FR-MG-AUTO-01 | Must Have | **P1** | `LEANKG_SKIP_FRESHNESS_CHECK=1` skips MCP auto-index (mega-graph OOM escape); document RocksDB mtime mismatch |
| FR-OPS-EMBED-CPU | Must Have | **P1** | Embed + MCP compose: `cpus: "6"`, `mem_reservation: 3g`; MCP `mem_limit: 6g` for mega-graphs (~147k vectors) |
| US-EMBED-04 / FR-EMBED-RESUME-05/06 / REL-052 | Must Have | P0 | Close further with 3-workspace + MCP vector-count evidence |
**New content:** Section **3.14** US-SEM-05; Section **5.15** FR-SEM-06; Section **5.17** FR-MG-AUTO-01 + FR-OPS-EMBED-CPU.
### v3.7.2-embed-resume - Day-2 embed resume is P0 (2026-07-18)
> **Problem:** Operators re-running embed against a **persisted RocksDB volume** (standalone `embed --wait`, or turning embed on inside Docker MCP via `LEANKG_EMBED_BACKGROUND` / `LEANKG_EMBED_ON_BOOT` / setup scripts) still burn full cold-embed resources (ONNX + HNSW rebuild) as if starting fresh — even when most/all vectors are already stored. Mega-graphs cannot afford a second full pass.
> **Product intent (universal rule):**
> - **No embed data for project** → cold / fresh fill (first run).
> - **Embed data exists** → **always resume** (skip `fresh`, embed only missing/stale/`content_hash`-changed).
> Applies to **every** Docker/CLI path that starts embed — not only standalone `--wait`. Zero-dirty runs must exit quickly without dropping HNSW. Interrupted runs must resume without discarding already-`fresh` rows. Container restart with the same named RocksDB volume must **never** imply a wipe.
> **Relationship to FR-HNSW-E:** Incremental filter + `embedding_state` already exist in code and were marked DONE, but **day-2 resource behavior is incomplete** (notably unconditional HNSW drop/rebuild; no mid-run checkpoint; full index can stale everything). Tracker sets `FR-HNSW-E` → **PARTIAL** and adds **Must Have** resume FRs as **Focus P0**.
**Decision table (Must Have):**
| Existing embed data in RocksDB / `.leankg`? | Docker / CLI starts embed? | Required behavior |
|--------------------------------------------|----------------------------|-------------------|
| **No** (empty / first project) | Yes | **Cold/fresh** fill |
| **Yes** | Yes (any path below) | **Resume** — skip `fresh`; delta only |
| **Yes** | Container restart, embed off | Leave data alone; MCP serves existing HNSW |
| Explicit `--full` / `LEANKG_EMBED_BACKGROUND_FULL=1` / `LEANKG_FORCE_REINDEX=1` | Yes | Intentional full rebuild (ops escape hatch only) |
**Covered entry paths (all must obey the table):**
1. Standalone: `docker run … embed --wait --project <container-path>`
2. In-process MCP: `LEANKG_EMBED_BACKGROUND=1` after `mcp-http` bind
3. Legacy foreground boot: `LEANKG_EMBED_ON_BOOT=1` / `embed_if_needed` in `entrypoint.sh`
4. Setup: `LEANKG_DOCKER_SETUP=1` offline embed before MCP
5. `scripts/docker-up.sh` index + embed flow
**Success metrics (day-2 KPIs):**
| KPI | Target | Measurement |
|-----|--------|-------------|
| Second embed on unchanged graph (any path) | Near-zero ONNX batches; wall time **minutes not hours** on mega-graph | Log `skipped_fresh` ≈ work list; `embedded` ≈ 0 |
| Zero-dirty run | **No** HNSW drop + rebuild | Log / unit: skip path when `to_embed` empty and no orphans |
| Interrupted then restarted | Resume; already-`fresh` rows not re-inferred | Kill mid-run → re-run; `embedded` ≤ remaining dirty |
| RocksDB volume remount / MCP re-enable embed | Vectors + state survive; resume not wipe | Same named volume; second start sees prior state |
| Empty project first embed | Cold fill allowed | No existing `embedding_state` / vectors |
**New content:**
- Section **3.15** — US-EMBED-01..04
- Section **5.16** — FR-EMBED-RESUME-01..06 + REL-052; note on FR-HNSW-E PARTIAL
- Section **8.5** — v3.7.2 embed-resume release gate
- Section **9** — NFR rows for day-2 embed
- Tracker: Focus **P0** open queue = embed-resume (VE gate remains DONE)
**Standalone operator pattern (placeholders only — never commit personal host mounts):**
```bash
docker run --rm \
--cpus 4 --memory 10g \
-v leankg_leankg-rocksdb:/data/leankg-rocksdb \
-v leankg_leankg_models:/root/.cache/leankg \
-v "$PWD":/workspace \
-v /Users/you/work/other-repo:/workspace-other \
-e LEANKG_DB_ENGINE=rocksdb \
-e LEANKG_ROCKSDB_ROOT=/data/leankg-rocksdb \
-e LEANKG_EMBED_FAST=1 \
-e LEANKG_EMBED_MODEL=bge-q \
-e LEANKG_EMBED_MAX_SEQ=128 \
-e LEANKG_EMBED_MAX_BLOB_CHARS=500 \
-e LEANKG_EMBED_MAX_MB=0 \
-e OMP_NUM_THREADS=1 \
-e RUST_LOG=leankg=info \
leankg-leankg:latest \
embed --wait --project /workspace-other --workers 8 --batch-size 128 --types function,method
```
Second identical run (unchanged code) **must** skip fresh rows and must **not** pay cold-embed cost.
**Docker MCP pattern (embed turned on later — still resume):**
```bash
# Same named RocksDB volume as a prior embed. Turning embed on must NOT wipe.
# e.g. LEANKG_EMBED_BACKGROUND=1 in compose / .dockerfile — incremental only.
```
### v3.7.1-sem-mcp-enhance - Semantic MCP live verification → later enhancements (2026-07-17)
> **Evidence:** [`docs/semantic-search-mcp-verification-2026-07-17.md`](semantic-search-mcp-verification-2026-07-17.md) — Docker HTTP MCP (`project=/workspace`), RocksDB index populated. **No code changes required** for correctness; this revision captures **product enhancements** for a later sprint.
**Baseline confirmed GREEN (do not reopen as bugs):**
- `semantic_search` → `method: hnsw+rerank`; ANN distance and rerank scores agree on ranking direction
- `concept_search` → ontology match + code refs; `kg_self_test` → `all_ok: true` (schema arities canonical)
- `kg_semantic_context` → seed + 1-hop graph traversal recovers after one transient socket drop
- Contrast: `explain_node` (graph-shaped) vs `search_code` name path (flat) both useful
**Enhancement backlog (later — MoSCoW):**
| ID | Priority | Problem observed | Product intent |
|----|----------|------------------|----------------|
| US-SEM-01 / FR-SEM-01 | Should Have | Top-level `tokens` = *delivered*; `_token_budget.actual` was 3–4× for truncated tools | Honest dual accounting so agents budget correctly |
| US-SEM-02 / FR-SEM-02 | Should Have | `concept_search` / `kg_semantic_context` hit default `max: 1000` while siblings use 2–4k | Explicit per-tool budgets for ontology-heavy tools |
| US-SEM-03 / FR-SEM-03 | Should Have | One transient HTTP socket drop on long semantic call | Retry / keep-alive / connection hygiene |
| FR-SEM-04 / REL-051 | Should Have | Live MCP probe is ad-hoc | Formal live smoke checklist as release *complement* to `cargo test --features embeddings` |
| US-SEM-04 / FR-SEM-05 | Could Have | Top-10 `semantic_search` can collapse to one file (8/10) | Optional file-diversity / MMR post-filter |
**New content:**
- Section **3.14** — US-SEM-01..04
- Section **5.15** — FR-SEM-01..05 + REL-051
- Section **9** — NFR rows for token honesty + MCP HTTP flake resilience
- Tracker: Focus **P2/P3** open items (do not displace P1 Must Have queue)
### v3.7.0-vector-engine - Optimized Local-First Vector Graph Engine (2026-07-17)
> **Task inventory move (same day):** All US/FR/Release status tables and checkboxes were moved to [`prd-task-tracker.md`](prd-task-tracker.md). Sections 3/4/5/8 now reference that file instead of duplicating lists.
> **Code status (synced 2026-07-17 — PR [#80](https://github.com/FreePeak/LeanKG/pull/80) `feature/vector-engine-gate`):** P0 Vector Engine **DONE** — unit (56), e2e (`tests/vector_engine_e2e.rs`), bench (`cargo bench --bench vector_engine_ab`), CI-sim `cargo test --lib` (651). Measured A/B: token **−65.0%**, tool **−84.6%**, speedup **2.50×** (100 tasks). 1M ANN P95≈**0.055ms** (Neon). Idle RSS: lean-bench absolute ≈**65MB** / warm **delta ≈58MB** (unit/e2e assert `delta_ok` — absolute process RSS is not CI-safe under debug `cargo test --lib`). TTC P95≈**0.068ms**. `LEANKG_VE_GATE_FULL=1` → `ready_for_default=true` / `preferred_ann_backend=local_engine`. Report: [`docs/benchmarks/vector_engine_gate_results.json`](benchmarks/vector_engine_gate_results.json). Cozo remains runtime default until callers honor the gate. Crate **0.19.0**. Awaiting merge to `main`.
> **Mission reinforcement:** *"Stop Burning Tokens. Start Coding Lean."* Surgical retrieval = Semantic Search (vectors) + Structural Graphs (LSP/KG). Same product surface as FR-HNSW-*; **new storage/runtime engine** for constrained local hardware and cloud scale without rewriting core query logic.
**Strategic decision (relationship to v3.6.2 / v3.6.3):**
- **Keep** CozoDB `::hnsw` on `embedding_vectors:vec_idx` as the **current shipped canonical ANN** (FR-HNSW-B) until the Local/Cloud vector engine reaches parity and FR-VE-GATE flips default.
- **Adopt** a decoupled **3-tier storage architecture** (graph topology + quantized RAM vectors + flat payload) as the **next-gen LocalEngine / CloudEngine** path — solves query latency, idle RAM, and SSD write amplification under M2 Pro / 16GB / 256GB SSD constraints; scales to Linux x86_64 + TiKV without rewriting retrieval APIs.
- **Do not** reopen FalkorDB/Redis as cold-embed SLA fixes (v3.6.3 Won't Do still stands). This track is about **query/runtime I/O + memory**, not ONNX cold-write throughput.
**Success metrics (product KPIs):**
| KPI | Target | Measurement |
|-----|--------|-------------|
| Token consumption vs grep/cat baseline | ≥ **61%** reduction (floor **60%**) | Agent A/B (`run_kilo_ab_final.sh` / existing benchmark) |
| Tool-call frequency vs baseline | ≥ **84%** reduction (floor **80%**) | Same A/B harness |
| Task success rate | ≥ baseline | Patch/tests pass without hallucination regression |
| Time-to-resolution | ≥ **2×** faster than baseline | End-to-end task timer |
| Idle daemon RSS | **< 150MB** | Local MCP idle after warm |
| Time-to-context (P95) | **< 100ms** | JSON chunks + deps payload to agent |
| ANN query P95 (1M SQ8, local) | **< 50ms** | `cargo bench` |
| Recall @ efSearch=50 vs FP32 brute-force | **> 90%** | Bench + unit |
| Disk reads / page faults vs legacy mmap | ≥ **80%** reduction | Bench instrumentation |
| 2GB cgroup survival | Never OOM-killed | Simulated cgroup test |
**New content:**
- Section **3.13** — US-VE-01..08 (vector engine stories)
- Section **5.14** — FR-VE-* (3-tier storage, SIMD, HNSW prune, dual-write, GC, tests/benches)
- Section **6.10** — HLD for LocalEngine vs CloudEngine + 3-tier diagram
- Section **8.4** — v3.7 vector-engine release gate
- Section **9** — NFR table refreshed for idle/query/hardware targets
### v3.6.3-embed-runtime - Cold embed SLA reality + MCP decoupling (2026-07-16)
> **Measured reality (mega-graph cold embed):** end-to-end sustained rate is ~**170 vec/sec** → ~**36 min** for ~371k `function,method` nodes (M2 Pro 10c). Writer-only microbenches on empty RocksDB show Cozo `import_relations` at ~**100k–130k vec/sec** (<1 min for 371k). **Storage commit / WAL is not the cold-SLA bottleneck**; ONNX inference + end-to-end CPU contention is.
**Done (ops / architecture):**
- MCP boot decoupled from embed: `LEANKG_EMBED_ON_BOOT=0` + in-process `LEANKG_EMBED_BACKGROUND=1` (shared `CozoDb`). MCP healthy ~60s while embed continues. See FR-EMBED-R1.
- Parallel embed pipeline + `import_relations` + `DirectEmbedder` (FR-EMBED-R2). ~2× vs earlier `:put` path (~73 min → ~36 min ETA) — still above aspirational <10/<20 min cold.
**Tried and rejected as cold-SLA fixes (evidence in `generated_docs/embed_bg_job_and_runtime_plan_2026-07-15.md`):**
- Cozo RocksDB WAL-off / `sync(false)` / no-snapshot write txs (`LEANKG_COZO_ROCKS_BULK`) — **≤1.15×** writer-only; no meaningful e2e gain.
- Redis Stack HNSW as vector side-store (`LEANKG_EMBED_VECTOR_STORE=redis`) — bulk HASH write ~164k/s (similar to Cozo); live HNSW during write ~2.7k/s (**worse**). Does **not** beat Cozo for cold SLA. Keep Cozo HNSW as canonical (FR-HNSW-B). Redis remains experimental only.
**Product SLA (revised):**
- **Must:** MCP never blocks on cold embed; semantic tools degrade until HNSW ready; day-2 incremental embed stays fast (FR-HNSW-E).
- **Aspirational / open:** cold functions-only <20 min on ~371k (needs **faster inference / smaller model / less volume**, not a new DB). Do not plan FalkorDB/Redis migration to fix cold embed.
**New FRs:** Section **5.12** additions FR-EMBED-R1..R4.
### v3.6.2-hnsw-semantic - Drop LSH roadmap; expand CozoDB HNSW for semantic search (2026-07-15)
> **Strategic decision:** LeanKG differentiates on **meaning-based retrieval** (dense embeddings + CozoDB native `::hnsw`), not on copy-paste / near-clone detection (MinHash / LSH). Agents need “what means like this,” not “which bodies are Jaccard-near.”
**Cancel / Won’t Do (LSH track):**
- FR-LSH-A..F and FR-BENCH-A (CBM MinHash parity) — **Won’t Do**. Do not expand MinHash/LSH; do not adopt Cozo `::lsh` for clones either (clone ANN is out of product focus).
- Custom in-process LSH (`src/minhash.rs` + `find_clones --cross-file`) **removed** on `integration/prd-pending` (FR-HNSW-A).
- Same-file Jaccard `find_clones` MCP + `leankg clones` CLI **hard-removed** (2026-07-20) — non-strategic and unusable on mega-graphs (`max_functions` guard). Prefer `semantic_search` / `concept_search` for discovery.
- US-CBM-B7 / FR-B30 / FR-B31 remain historically DONE for the light same-file Jaccard tool; product surface no longer exposes it.
**Adopt / Expand (HNSW track) — reuse CozoDB 0.7.x native index (already in tree):**
- LeanKG already depends on `cozo = "0.7.6"` and already uses `::hnsw create embedding_vectors:vec_idx` (`src/embeddings/state.rs`, `src/retrieval/pipeline.rs`). Pattern to double down on: **LeanKG extracts features → Cozo indexes**.
- New FRs: Section **5.12** (HNSW expansion) + Section **5.13** (LSP-only remainder from former CBM adoption track).
- **Implementation landed on `integration/prd-pending` (2026-07-15):** FR-HNSW-A..F + FR-BENCH-HNSW + US-CBM-C1 / FR-C01 (Docker `--features embeddings` + `entrypoint.sh` `embed_if_needed`; HNSW `semantic_search` dispatch; `LEANKG_HNSW_{M,EF_CONST,EF}` knobs; `tests/hnsw_recall_e2e.rs` synthetic recall@k smoke).
- **PRD hygiene (2026-07-15):** corrected language / Graphify / MemPalace status rows that overclaimed “DONE” for extractors that exist as modules but are **not hooked into the index walk** (Swift, Vue/Svelte, SQL DDL). Softened “17 languages fully extracted” claims to match `find_files_sync` + `get_language`.
- Research record: `generated_docs/research_cozo_native_lsh_vs_custom_minhash_2026-07-15.md` (main tree) — Cozo already ships both `::hnsw` and `::lsh`; we choose HNSW only.
**CBM deep-compare (v3.6.1) still valid for LSP gaps** (FR-LSP-A..D). MinHash / LSH “wins” from that compare are explicitly **not** adopted.
### v3.6-lsp-ontology - LSP infra, language breadth, status flips (integration/prd-pending push)
- LSP infrastructure shipped (US-CBM-B1 infra, FR-B03..B07 scaffolding): new `src/lsp/{bridge,client,config,mod}.rs` — generic JSON-RPC bridge that spawns any configured language server, answers `textDocument/definition` and `/references`; per-(language, workspace_root) client cache; 12-language manifest detection (go.mod / package.json / Cargo.toml / pyproject.toml / pom.xml / build.gradle* / tsconfig.json / Gemfile / mix.exs / pubspec.yaml / Project.toml / Package.swift). Wired through MCP `resolve_with_lsp` (`src/mcp/handler.rs:1674`) and CLI `leankg lsp-resolve` (`src/main.rs:lsp_resolve`). Commits `534cd7f` + `64b0fa6`.
- `typed_resolve` feature flag landed (US-CBM-B10 / FR-B08, `8971dc5`). Default `LspConfig` is still empty (`src/lsp/config.rs:57`); LSP server bootstrap (default `lsp:` block for gopls + tsserver + pyright) remains the open follow-up.
- Codebase version: 0.17.8 → 0.17.9 (`3e103b1 chore(release): regen Cargo.lock for 0.17.9` + `1c6f1eb chore(release): bump version to 0.17.9`).
- Language breadth — **status corrected 2026-07-15 (wiring audit):**
- US-LANG-01 Dart — **DONE and indexed** (in `find_files_sync` + `get_language`) (`7ec6484`)
- US-LANG-02 Swift — **PARTIAL**: regex extractor in `src/indexer/swift.rs` (`7027d6b`); **not wired** into `find_files_sync` / index walk (`.swift` not scanned)
- US-LANG-03 XML — **DONE and indexed** (`.xml` + Android path) (`92db9aa`)
- US-GF-10 Vue/Svelte — **PARTIAL**: regex extractors in `src/indexer/sfc.rs` (`e617a49`); **not wired** into index walk (`.vue` / `.svelte` not scanned)
- US-GF-12 SQL DDL — **PARTIAL**: parser in `src/indexer/sql.rs` (`de314eb`); **not wired** into index walk (`.sql` not scanned)
- Agent-graph UX series — DONE:
- US-GF-07 rationale extraction (`# WHY:` / `# NOTE:` / `# HACK:` / `# FIXME:` / `# XXX:` markers) → `rationale` elements with `explains` edges (`b0c9477`)
- US-GF-08 PR impact dashboard — `get_pr_impact` MCP + `leankg prs` CLI (`30e41f0`)
- US-GF-09 work-memory reflect loop — `report_query_outcome` + `.leankg/reflections/LESSONS.md` (`373e808`)
- US-GF-11 portable graph snapshot — `export_graph_snapshot` MCP (`0087991`)
- MemPalace series — DONE:
- US-MP-01 temporal knowledge graph — `valid_from` / `valid_to` on `Relationship` (`bc9cc53`)
- US-MP-04 specialist agent contexts — `agent_focus` + `agent_diary_{read,write}` MCP (`1ea4bcd`)
- US-MP-05 consistency checker — `check_consistency` MCP + `leankg check-consistency` CLI (`60a6111`)
- US-MP-06 cross-domain tunnels — `find_tunnels` MCP + `leankg tunnels` CLI (`5b6547e`)
- CBM structural — DONE:
- US-CBM-B6 event-channel edges `emits` / `listens_on` (`25a3b37`)
- US-CBM-B7 clone / near-duplicate detection — historically shipped as `find_clones` MCP + `leankg clones` CLI (`55e6e72`); **hard-removed 2026-07-20** (prefer semantic HNSW discovery)
- US-CBM-B8 cross-repo similar edges — `find_cross_repo_similar` (`ab16c9b`)
- US-CBM-C2 hot-path cache for high-frequency MCP tools (`836f0a3`)
- GitNexus — DONE:
- US-GN-07 `get_cluster_skill` MCP — per-cluster `SKILL.md` (`10b15a0`)
- US-GN-08 `get_overview_context` MCP — resource-style overview (`9124959`); formal `resources/read` not yet wired (PARTIAL).
- Team / distribution — DONE:
- US-14 npm-based installation wrapper (`df0fec2`)
- US-V2-11 CI/CD auto-graph update — GitHub Actions workflow that reindexes / commits the portable snapshot on release (`eb3d331`)
- US-V2-12 `get_team_map` MCP — team + on-call ownership + environment map (`3368b5f`)
- Quality gate: `cargo fmt --all -- --check`, `cargo clippy --release --all-targets -- -D warnings`, `cargo test --release --lib` (496), `cargo test --release --bin leankg` (491), `cargo test --release --test ontology_e2e` (16/16) all PASS (`docs/implementation/prd-integration-2026-07-14.md`).
- MCP tool count: 65 → 85 (`src/mcp/tools.rs` — audit using `awk '/^[[:space:]]+name:[[:space:]]*"/{ print }' src/mcp/tools.rs | sort -u | wc -l` = 85 unique tool registrations as of 2026-07-14).
- Open follow-ups: default `lsp:` block for gopls + tsserver + pyright + dart-language-server + sourcekit-lsp + kotlin-language-server; FR-B03 / FR-B04 actual `typed` resolution for Go and TS; FR-MG-03 single-repo root expansion; 3D graph UI (Track E). **Superseded by v3.6.2 for LSH:** do not pursue FR-LSH-*; pursue FR-HNSW-* instead.
### v3.6.1-cbm-deep-compare - In-process read of CBM LSH + Hybrid LSP (2026-07-15)
> Source: direct read of `DeusData/codebase-memory-mcp` at `/Users/linh.doan/work/harvey/freepeak/codebase-memory-mcp` (v0.9.x, Pure C, 158 languages, 15 MCP tools).
>
> **Superseded for LSH:** v3.6.2 cancels MinHash/LSH adoption. This section remains as competitive research only. **Still actionable:** Hybrid LSP gaps → FR-LSP-A..D in Section 5.13.
>
> **TL;DR — CBM's "Hybrid LSP" is not actual LSP.** It is a lightweight C implementation of language type-resolution algorithms embedded in the binary (no spawn, no JSON-RPC). Their `LshIndex` for near-clones is a textbook MinHash+LSH pipeline — useful for *their* clone-edge product; **LeanKG will not mirror it** (semantic HNSW focus instead).
**CBM MinHash / LSH for `SIMILAR_TO` (clone) edges** — research only (`src/simhash/minhash.{h,c}`). Historical LeanKG comparison to `src/minhash.rs` is obsolete once that module is removed (v3.6.2).
| Knob | CBM | LeanKG (pre-removal) | Note |
|------|-----|----------------------|------|
| Role | Core clone product | Historical LeanKG helper (removed) | **Won't adopt** further LSH |
| Shingle unit | AST leaf trigrams `I/S/N/T` | Whitespace 5-grams | Irrelevant under HNSW strategy |
| Index home | In-process C | Custom Rust `LshIndex` (also unused Cozo `::lsh`) | Prefer deleting custom LSH; do not wire Cozo `::lsh` |
**CBM Hybrid LSP (pass over tree-sitter)** — `internal/cbm/lsp/{py,go,ts,java,kotlin,rust,c,cpp,cs,php,perl}_lsp.{c,h}` plus `type_rep.{c,h}`, `scope.{c,h}`, `type_registry.{c,h}`, `py_builtins.c`, `kotlin_builtins.c`, `rust_cargo.c`, `rust_proc_macros.c`, `rust_rustdoc.{c,h}`, `generated/python_stdlib_data.c` (12k lines of pre-baked stdlib metadata):
| Surface | CBM | LeanKG (`src/lsp/{bridge,client,config,mod}.rs`) |
|---------|-----|--------------------------------------------------|
| Approach | **In-process C type evaluator.** No `fork`/`exec`/`popen`, no JSON-RPC. Each language file re-implements the resolver inline (e.g., `py_lsp_init` / `py_lsp_process_file` / `py_lsp_bind_imports`) | **Real JSON-RPC bridge.** Spawns external server (`gopls`, `tsserver`, `pyright`, …); sends `textDocument/definition` + `/references`; caches one client per `(language, workspace_root)` |
| Languages | 10 — Python, TS/JS/JSX/TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, Perl (per-language files in `internal/cbm/lsp/`) | 12 manifest-detected (go.mod / package.json / Cargo.toml / pyproject.toml / pom.xml / build.gradle* / tsconfig.json / Gemfile / mix.exs / pubspec.yaml / Project.toml / Package.swift), **0 default-configured servers** |
| Setup | Zero. Embedded in the static binary | User must populate `lsp.servers.<lang>.command` in `leankg.yaml` |
| Correctness model | Re-implements the algorithm the way gopls/pyright/Roslyn would — output is "structurally compatible" | Uses the real server's answer; can get accurate types the C reimplementation misses |
| When does it run? | Per-file during extraction, BEFORE `CALLS` edges are written — refines `CALLS`/`USAGE`/`RESOLVED_CALLS` directly | After index, on demand via `resolve_with_lsp` MCP / `leankg lsp-resolve` CLI; has not yet been wired to write `resolution_method=typed` edges |
| Failure mode | Falls back to "textual resolution" (tree-sitter-only) for unsupported languages | Returns `Ok(None)` and the caller falls back to tree-sitter typed resolve (FR-B07) |
**LeanKG wins (what CBM does not have):**
- 85 MCP tools vs CBM's 15
- Ontology / concept / workflow layer
- **CozoDB native HNSW embeddings path** (semantic ANN) — primary differentiation going forward (v3.6.2)
- `env` namespacing + incident knowledge + service context + env-conflict detection
- Android / Kotlin / XML deep features, Graphify-inspired work-memory loop, tunnel detection, consistency checker, portable graph snapshot, npm install
- Real language-server correctness (when a server is configured)
- REST API + RocksDB multi-project HTTP team deploy
- Per-cluster SKILL.md, overview-context, team-map
**CBM wins — adopt vs ignore:**
- **Adopt:** Zero-setup Hybrid LSP on 10 languages → FR-LSP-A..D (Section 5.13)
- **Ignore (v3.6.2):** AST-trigram MinHash, K=64 signatures, big-bucket guards, clone Jaccard defaults — clone LSH is not LeanKG's bet
**Adoption FRs:** LSP → Section 5.13 (FR-LSP-A..D). HNSW expansion → Section 5.12 (FR-HNSW-*). Former FR-LSH-* → Won't Do.
### v3.5-unified - Single PRD+HLD document
- Merged `docs/requirement/prd-leankg.md` (v2 team infrastructure) → Section 3.12 / 5.11
- Merged `docs/design/hld-leankg.md` → Section 6.4–6.9 (HLD)
- Merged `leankg update` CLI PRD → US-UPD-01
- Confirmed CBM structural parity already in 3.11 / 5.10; deleted redundant source files
- Codebase status refresh: env/incident/service tools, vacuum scheduler, `kg_self_test`, `leankg update` marked DONE where implemented
- Removed: `docs/requirement/prd-*.md`, `docs/design/hld-leankg.md`, `docs/LeanKG_v2_PRD.html`, duplicate `.docs` PRD stubs
### v3.4-cbm-structural-merge - Merge structural parity (CBM) PRD + codebase status refresh
- Merged CBM structural parity into this document (Section 3.11 US-CBM, Section 5.10 FR-CBM); source file removed in v3.5
- Codebase audit (2026-07-13, v0.17.8): **65 MCP tools** in `src/mcp/tools.rs`; Phase 1 aggregators DONE; Routes/HTTP_CALLS extractors DONE; typed resolve / clones / cross-repo / 3D UI still PENDING
- Updated executive metrics, pending table, and roadmap Phase 1 status to match code
- Source CBM PRD retained as archive with pointer to this document as SoT
### v3.3-graphify-parity - Graphify competitive enhancements
- Competitive analysis of [Graphify](https://github.com/Graphify-Labs/graphify) (v8 / ~83k stars) vs LeanKG deploy + agent tooling
- Full comparison: `docs/analysis/graphify-comparison-2026-07-13.md`
- Added US-GF-01..12 user stories for Graphify-inspired agent graph UX, edge provenance, reports, PRs, and learning loop
- Added FR-GF-01..20 functional requirements (Section 5.9)
- Priority focus: shortest-path / explain / NL subgraph query, EXTRACTED|INFERRED|AMBIGUOUS edge labels, god-node ranking, GRAPH_REPORT.md — not rewriting LeanKG's stronger RocksDB multi-project deploy
### v3.2-toon-format - TOON response format for MCP tools (~40% token reduction)
- Added US-TOON-01 user story for TOON (Token-Oriented Object Notation) format adoption
- TOON is a compact notation that reduces field name repetition in arrays
- Example: `elements[2]{qualified_name,type}: src/main.rs::main,function` vs JSON with full field names
- TOON spec: https://github.com/toon-format/toon
- Added Section 7.5 TOON Response Templates for all MCP tool categories
### v3.1-massive-graph - Massive graph service expansion
- Added US-MG-01..05 user stories for service node double-click behavior
- Added FR-MG-01..08 functional requirements for expand-service optimization and filter UI
- Expand-service API optimized: targeted folder query (7.7k vs 1.5M elements), ~30% faster
- FR-MG-01..02, 04..08 implemented: expand-service returns all edge types, double-click calls expandService directly, filter panel always shows all 14 types, defaults = Service/Folder/File/Function
- FR-MG-03 (single-repo root expansion) still pending
### v3.0-consolidated - Full codebase audit
- Deep dive codebase analysis: 35 MCP tools verified (0 stubs), 28+ CLI commands, 10 language extractors
- Updated language support: 10 fully extracted (Go, TS/JS, Python, Rust, Java, Kotlin, C++, C#, Ruby, PHP) + 3 parser-only (Dart, Swift, XML) — **superseded by 2026-07-15 wiring audit:** only Go/TS/JS/Python/Rust/Java/Kotlin/Dart (+XML/TF/CI) are in the current index walk; C++/C#/Ruby/PHP/Swift not scanned
- Updated all user story statuses based on actual implementation
- Added missing feature sections: Git Hooks, Context Metrics, REST API, Wiki Generation, Global Registry, Graph Export, Orchestrator
- Unified RTK Compression status: ResponseCompressor (FR-RTK-11..15) now marked DONE
- Fixed US-GN-03 (Global Registry) status: DONE (was PENDING)
- Fixed AB Testing stories: US-AB-02..04 marked DONE
- Removed outdated references to non-existent features
- Added new user stories for recently implemented features
### v2.0-consolidated - Merged from 3 source PRDs
- Source 1: `prd-leankg.md` (v1.7, 2026-03-27)
- Source 2: `prd-leankg-v2.0-enhancements.md` (v2.0, 2026-03-27)
- Source 3: `prd-leankg-gitnexus-enhancements.md` (v1.0, 2026-03-27)
---
## 1. Executive Summary
LeanKG is a lightweight, local-first knowledge graph solution designed for developers who use AI-assisted coding tools. The mission is *"Stop Burning Tokens. Start Coding Lean."* — resolve AI agent **context blindness** with surgical retrieval: **Semantic Search (vectors) + Structural Graphs (LSP/KG)**, not shotgun `grep`/`cat`.
Unlike heavy frameworks like Graphiti that require external databases (Neo4j) and cloud infrastructure, LeanKG runs on constrained local hardware (Apple Silicon, 16GB RAM, 256GB SSD) with a strict idle footprint, while the same core logic can scale to self-hosted cloud (Linux x86_64, TiKV) via a storage abstraction — without rewriting retrieval APIs.
**Value proposition (agent economics):**
- Cut LLM tokens by ≥ **61%** and tool calls by ≥ **84%** vs traditional grep/cat baselines, while holding Fix Success Rate ≥ baseline
- Deliver code chunks + dependencies JSON to the agent in **< 100ms P95**; idle MCP **< 150MB RSS**
- Prefer vector+graph scalpel over full-repo dumps (see Section 3.13 / 5.14)
### 1.1 Why LeanKG for the company (vs Graphify / competitors)
> **Audience:** engineering managers deciding which knowledge-graph / AI-context stack to standardize.
> **Full matrix:** [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md).
**One-line decision:** Graphify is an excellent **personal skill + portable HTML report**. LeanKG is the **company platform**: one shared index, multi-repo Docker, surgical MCP context, ops/traceability, and measured token economics — so every developer’s agent burns fewer tokens on the same monorepo every day.
| Decision criterion | LeanKG | Graphify | Why LeanKG wins at company scale |
|--------------------|--------|----------|----------------------------------|
| **$/day AI agent cost** | TOON/RTK + budgeted MCP; A/B ≥61% tokens / ≥84% tool calls vs grep/cat | Budgeted subgraph query; no durable compression stack | Savings compound across **N developers × N sessions/day** on a shared index |
| **Monorepo / mega-graph** | RocksDB multi-project; mega-safe search/query paths; mem budgets | `graph.json` + 5k HTML viz cap; NetworkX in-memory | Company monorepos (100k–600k+ elements) need a server, not a file |
| **Multi-repo team deploy** | `LEANKG_PROJECT_DIRS` + Docker HTTP MCP `:9699` + REST `:8080` | Shared HTTP over one `graph.json` | One container serves many mounts; resume embeds day-2 |
| **Agent depth** | ~85 MCP tools (impact, ontology, services, Android, docs↔req, PR, reflect) | ~9–10 MCP tools | Agents solve blast-radius / env / incident / req-trace without reinventing |
| **Trust / ops** | Severity-graded impact, incidents, env promotion, service_calls | Edge confidence tags (EXTRACTED/INFERRED) — LeanKG must finish parity | Managers care about **change risk** and **production topology**, not only concept maps |
| **Human explorer** | Live ui-v2 (Force/Tree/Circles) over REST | Static `graph.html` | Live index stays fresh with watch/auto-index |
| **Where Graphify still teaches us** | Packaging: always-on install, report/HTML artifacts, honest-edge UX | Skill install + `GRAPH_REPORT.md` + vis.js share | Close these in **P1 queue** below — do **not** chase multimodal |
**Ordered company-adoption queue (Focus P1 — after P0 procedural auto-update):**
1. **US-COST-01** — Publish manager ROI brief (tokens + tool calls + multi-repo TCO)
2. **US-GF-14** — Three-verb narrative (path · explain · query) so agents pick cheap tools first
3. **US-GF-17** — Always-on graph-first install/hooks (primary **cost lever**: agents stop grepping first)
4. **US-GF-04** — Honest edges in MCP + ui-v2 (trust = adoption)
5. **US-GF-06** — Auto `GRAPH_REPORT.md` (onboarding without 85-tool wall)
6. **US-GF-13** — Bounded HTML export (share in PRs without replacing live UI)
7. **US-UI2-06** — NL Query FAB (humans get the same cheap verb)
8. **US-UI2-07** — ui-v2 cutover (one default explorer for the company)
**P0 first (v3.7.9):** **US-ONT-PROC-01** — procedural ontology auto-update while using (see §3.18 / §5.21). Without this, `kg_trace_workflow` stays a stale boot-time artifact.
**Explicit non-goals for company ROI:** PDF/image/video graph ingest; replacing CozoDB with NetworkX; chasing 36 languages before typed resolve depth.
**Key Metrics (v0.19.0 — codebase `origin/main` 2026-07-17; engine KPIs in Section 9 / 8.4):**
- **Vector engine (v3.7 P0):** `src/vector_engine/*` — P0 gates **DONE** on `feature/vector-engine-gate`; A/B −65.0%/−84.6%/2.50×; opt-in `LEANKG_VECTOR_ENGINE`; Cozo default until callers honor `preferred_ann_backend()`
- **85 MCP tools** defined in `src/mcp/tools.rs` (stdio + HTTP/SSE)
- 30+ CLI commands (added `leankg lsp-resolve`, `leankg check-consistency`, `leankg tunnels`, `leankg prs`, `leankg reflect`; `leankg clones` hard-removed 2026-07-20)
- **Indexed languages (production walk):** Go, TS/JS, Python, Rust, Java, Kotlin, Dart + Android/XML + Terraform/CI YAML + common config manifests. **Extractor modules exist but not indexed yet:** Swift (`swift.rs`), Vue/Svelte (`sfc.rs`), SQL DDL (`sql.rs`). Parsers may exist for Ruby/PHP/etc. without index-walk wiring. + Markdown docs
- 8 compression/read modes + TOON responses
- Smart orchestrator with persistent cache + hot-path cache for high-frequency MCP tools (`836f0a3`)
- Git hooks (pre-commit, post-commit, post-checkout) + CI/CD auto-graph update GitHub Actions workflow (`eb3d331`)
- REST API server with auth
- Context metrics tracking
- Global multi-repo registry
- RocksDB multi-project HTTP deploy
- Structural aggregators: `get_architecture`, `get_graph_schema`, `find_dead_code` (DONE)
- Route + `http_calls` extractors for Go (chi/gin/echo) and TS (express/fastify) (DONE)
- Event-channel edges `emits` / `listens_on` (DONE `25a3b37`)
- Wake-up context + consistency checker + cross-domain tunnels (`wake_up`, `check_consistency`, `find_tunnels` — DONE)
- LSP bridge infrastructure + `resolve_with_lsp` MCP + `leankg lsp-resolve` CLI (DONE `534cd7f` + `64b0fa6`); `typed_resolve` feature flag in `IndexerConfig` (DONE `8971dc5`); **default `LspConfig::servers` is still empty** — default-server bootstrap is the open follow-up.
- Call edges carry `resolution_method` + numeric `confidence` (`name` / `name_file_hint` / `unresolved`; `typed` not yet produced)
- Temporal knowledge graph (`valid_from` / `valid_to`) + specialist agent contexts (`agent_focus` + diary) (DONE `bc9cc53`, `1ea4bcd`)
- Agent-side report: rationale nodes (WHY/NOTE/HACK/FIXME/XXX), PR impact dashboard, work-memory reflect loop → `.leankg/reflections/LESSONS.md` (DONE `b0c9477`, `30e41f0`, `373e808`)
- Portable graph snapshot (`export_graph_snapshot` MCP) (DONE `0087991`)
- npm-based installation wrapper (DONE `df0fec2`)
**Competitive notes:**
- vs [Graphify](https://github.com/Graphify-Labs/graphify): **company vs personal** — see §1.1 + §3.10 + [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md) (Jul-13 matrix: [`graphify-comparison-2026-07-13.md`](analysis/graphify-comparison-2026-07-13.md); many MCP “Missing” rows are now DONE)
- vs [codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp): see Section 3.11 / 5.10 — Lean into business-context depth; close structural gaps; do **not** chase 158-language / Pure-C parity
- vs LSP-by-default (CBM style): see Section 3.11 / 5.10 — LeanKG now has the bridge + wiring (FR-B03..B07 + FR-B08); `typed`-class edges still PENDING for Go (`FR-B03`) and TS (`FR-B04`).
- vs mmap-heavy / full-FP32-in-RAM vector stores: LeanKG targets SQ8 hot path + flat payload post-filter (Section 5.14 / 6.10) to protect 256GB SSDs and 16GB laptops.
---
## 2. Problem Statement
### 2.1 Current Pain Points
| Pain Point | Description |
|------------|-------------|
| **Context Window Dilution** | AI tools scan entire codebases, including irrelevant files, wasting context window tokens |
| **Outdated Documentation** | Manual docs quickly become stale; AI receives wrong context |
| **Business Logic Disconnect** | No clear mapping between business requirements and code implementation |
| **Token Waste** | Redundant code scanning generates unnecessary token costs |
| **Poor Code Generation** | AI lacks accurate context, producing incorrect or suboptimal code |
| **Feature Transfer Difficulty** | Onboarding new developers requires extensive code exploration |
| **Impact radius lacks confidence grades** | `get_impact_radius` returns all edges at equal weight; LLM cannot distinguish "WILL BREAK" from "MIGHT BE AFFECTED" |
| **No pre-commit risk signal** | No tool exists to assess change risk before commit |
| **Flat search results** | `search_code` returns symbol matches with no grouping by functional area |
| **No shortest-path / explain verbs** | *(closing)* MCP `shortest_path` / `explain_node` exist; product narrative + UI NL still incomplete — see US-GF-14 / US-UI2-06 |
| **Opaque edge provenance** | Agents cannot tell EXTRACTED vs INFERRED vs AMBIGUOUS at a glance (US-GF-04 / FR-GF-07..09 — **P1**) |
| **No architecture brief artifact** | Missing auto god-node + surprising-connection report (`GRAPH_REPORT.md`) after index (US-GF-06 / FR-GF-13 — **P1**) |
| **Agents still grep first** | Without always-on install/hooks, token savings never materialize at company scale (US-GF-17 — **P1**) |
| **No query outcome learning** | Context metrics exist; default skill loop for useful/dead_end still weak (US-GF-16 — P2) |
---
## 3. User Stories
### 3.1 Core MVP Stories (US-01 to US-18)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-01`..`US-18`.
### 3.2 v2.0 Enhancement Stories (US-19 to US-27)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-19`..`US-27`.
### 3.3 GitNexus Enhancement Stories (US-GN-01 to US-GN-09)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-GN-01`..`US-GN-09`.
### 3.4 AB Testing Stories (US-AB-01 to US-AB-05)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-AB-01`..`US-AB-05`.
### 3.5 RTK Compression Stories (US-RTK-01 to US-RTK-15)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-RTK-*`.
### 3.6 Infrastructure Stories (US-INF-01 to US-INF-10)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-INF-*`.
### 3.7 Additional Language Stories (US-LANG-01 to US-LANG-03)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-LANG-*`.
### 3.8 Massive Graph Stories (US-MG-01 to US-MG-05)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-MG-*`.
### 3.9 TOON Format Stories (US-TOON-01)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-TOON-01`.
**Detailed Feature Descriptions:**
<details>
<summary>US-TOON-01: TOON Response Format</summary>
**Problem:** JSON responses from MCP tools include repetitive field names in arrays, wasting tokens.
**Solution:** Adopt TOON (Token-Oriented Object Notation) format which omits field names within array items when they match the schema template.
**Example comparison:**
JSON (312 tokens):
```json
{
"elements": [
{"qualified_name": "src/main.rs::main", "type": "function", "language": "rust"},
{"qualified_name": "src/lib.rs::init", "type": "function", "language": "rust"}
],
"tokens": 312
}
```
TOON (187 tokens, 40% reduction):
```
{
elements[2]{qualified_name,type,language}:
src/main.rs::main,function,rust
src/lib.rs::init,function,rust
tokens: 187
}
```
**Specification:** https://github.com/toon-format/toon
**Implementation:**
- All MCP tool responses wrapped in Response Format Envelope
- Envelope: `{status: ok, tool: <name>, format: toon|json, tokens: <count>, data: <toon_string>}`
- Default format is `toon`; clients can request `json` via `format=json` parameter
</details>
**Detailed Feature Descriptions:**
<details>
<summary>US-MG-01: Service node loads all elements and edges</summary>
**Problem:** Previously, double-clicking a Service node in the graph only returned a subset of relationship types (`contains`, `defines`, `imports`, `calls`). Other edges like `extends`, `implements`, `references`, `tested_by` were missing from the expanded view.
**Behavior:**
- Double-click on a Service node → `/api/graph/expand-service?path=<absolute_path>` returns ALL elements under that service folder AND ALL relationship types between them
- Backend must NOT filter by relationship type — let the frontend filter UI control visibility
- All node types are returned: `service`, `folder`, `directory`, `file`, `module`, `class`, `struct`, `interface`, `enum`, `function`, `method`, `constructor`, `property`, `decorator`
**Backend changes:**
- `api_graph_expand_service` handler removes the `matches!(r.rel_type.as_str(), "contains" | "defines" | "imports" | "calls")` filter
- Returns ALL relationships where source is in the service folder
**Frontend changes:**
- Filter UI is the sole mechanism for controlling what's visible
- User toggles edge types on/off to see calls, imports, contains, etc.
</details>
<details>
<summary>US-MG-02: Single-repo full expansion</summary>
**Problem:** When a service has many nested folder layers (e.g., `platform-transport/be-engagement/internal/handler/v2/`), the user must double-click through each folder level to see contents. This loses the overall service context.
**Behavior:**
- Double-click on a Service node loads the ENTIRE service tree at once
- All folders, sub-folders, files, and functions are loaded in a single API call
- The filter UI controls visibility: by default, only `Service`, `Folder`, `File`, `Function` nodes are shown
- User can toggle on `Method`, `Class`, etc. to see more detail without making another API call
- For single-repo projects (no multi-service layout), the same behavior applies — the root is treated as the "service"
**Rationale:** Loading everything at once is fast (~13s for 7.7k elements after optimization) and avoids the UX problem of losing the chart context during multi-level drilling.
</details>
<details>
<summary>US-MG-03: Filter UI always shows all node types</summary>
**Problem:** Previously `discoveredNodeTypes` was computed from loaded graph data. If the current view only has `File` and `Function` nodes, the filter panel only shows `File` and `Function` toggles.
**Behavior:**
- The filter panel ALWAYS shows ALL node types from `DEFAULT_NODE_TYPE_ORDER`: `Service`, `Folder`, `Directory`, `File`, `Module`, `Class`, `Struct`, `Interface`, `Enum`, `Function`, `Method`, `Constructor`, `Property`, `Decorator`
- This is a static list, not data-driven
- Types not present in current data still appear but are visually dimmed or show "(0)" count
**Implementation:**
- `discoveredNodeTypes` in `App.tsx` uses `DEFAULT_NODE_TYPE_ORDER` directly instead of computing from `data.nodes`
</details>
<details>
<summary>US-MG-04: Default visible filters</summary>
**Problem:** Previously the default visible labels included `Service`, `Folder`, `Directory`, `File` — missing `Function` which is the most important code-level type. Also, ALL types started as visible, making the graph too noisy.
**Behavior:**
- **Default ON (visible):** `Service`, `Folder`, `File`, `Function`
- **Default OFF (hidden):** `Directory`, `Module`, `Class`, `Struct`, `Interface`, `Enum`, `Method`, `Constructor`, `Property`, `Decorator`
- `resetToStructuralDefaults()` resets to these 4 types
- After double-clicking a service, filters reset to these 4 defaults
**Implementation:**
- `DEFAULT_VISIBLE_LABELS` = `['Service', 'Folder', 'File', 'Function']`
- `useGraphFilters` initial state uses `DEFAULT_VISIBLE_LABELS`
- `resetToStructuralDefaults()` uses `DEFAULT_VISIBLE_LABELS`
</details>
<details>
<summary>US-MG-05: Expand-service API optimization</summary>
**Problem:** The original `api_graph_expand_service` handler called `g.all_elements()` and `g.all_relationships()` which loaded ALL 1.5M elements and ALL 1.6M relationships into memory, then filtered in Rust. This took ~19 seconds.
**Solution (DONE):**
- Added `get_elements_in_folder()` to `GraphEngine` using CozoDB `regex_matches(file_path, $pat)` with bound parameter
- Handler converts absolute paths to DB format: `/Users/.../be-engagement` → `./platform-transport/be-engagement`
- Only loads ~7.7k relevant elements instead of 1.5M
- Response time: ~13s (30% improvement). Remaining time is from loading all 1.6M relationships.
</details>
### 3.9 MemPalace-Inspired Stories (US-MP-01 to US-MP-08)
> **Source:** Competitive analysis of [MemPalace](https://github.com/milla-jovovich/mempalace) — the highest-scoring AI memory system on LongMemEval (96.6% R@5 raw mode). Key differentiator: raw verbatim storage without summarization, structured spatial navigation (wings/rooms/closets/drawers), temporal entity graph with validity windows, and a 4-layer memory stack (L0-L3) for token-efficient context loading.
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter IDs for this section (`US-*` / related).
**Detailed Feature Descriptions:**
<details>
<summary>US-MP-01: Temporal Knowledge Graph</summary>
**MemPalace inspiration:** Entity relationships have validity windows (`valid_from`, `valid_to`). When something stops being true, it's invalidated but retained for historical queries.
**LeanKG adaptation:**
- Add `valid_from` and `valid_to` (nullable) fields to `Relationship` table
- When re-indexing detects a removed import/call, set `valid_to = now()` instead of deleting
- New MCP tool: `temporal_query` — "what did the dependency graph look like before commit X?"
- New MCP tool: `invalidate_edge` — manually mark an edge as no longer current
- Timeline view: chronological story of how a code element's dependencies evolved
</details>
<details>
<summary>US-MP-02: Layered Context Loading (L0-L3)</summary>
**MemPalace inspiration:** 4-layer memory stack where L0+L1 (~170 tokens) are always loaded, L2 is on-demand, L3 is deep search.
**LeanKG adaptation:**
- **L0 — Project Identity** (~50 tokens): Project name, languages, top-level directories, architecture pattern.
- **L1 — Critical Facts** (~120 tokens): Module map, critical dependencies, recent change hotspots.
- **L2 — Cluster Context** (on demand): When a query touches a specific area, load the relevant cluster's symbols.
- **L3 — Deep Search** (on demand): Full graph traversal, impact analysis, cross-cluster queries.
- New MCP tools: `wake_up` (L0+L1), `load_layer` (L2/L3)
</details>
<details>
<summary>US-MP-03: Conversation/Decision Mining</summary>
**MemPalace inspiration:** Mines conversation exports (Claude, ChatGPT, Slack) to extract decisions, preferences, milestones. Stores raw verbatim.
**LeanKG adaptation:**
- New indexer module: `conversation_indexer` — parses Claude/ChatGPT/Slack export JSON
- Extracts: decisions, preferences, milestones, problems
- Creates `decision`, `preference`, `milestone`, `problem` element types
- Links decisions to code elements via `decided_about` relationship
- Store raw verbatim — no summarization
- New CLI command: `leankg mine-conversations ~/chats/ --format claude|chatgpt|slack`
</details>
<details>
<summary>US-MP-04: Specialist Agent Contexts</summary>
**MemPalace inspiration:** Define agent personas (reviewer, architect, ops) each with their own wing and diary.
**LeanKG adaptation:**
- Agent config in `.leankg/agents/*.json` — focus areas and context filters
- Each agent gets a filtered view of the graph
- Agent diary: per-agent CozoDB table storing session notes
- New MCP tools: `agent_focus`, `agent_diary_write`, `agent_diary_read`
</details>
<details>
<summary>US-MP-05: Contradiction & Staleness Detection</summary>
**MemPalace inspiration:** `fact_checker.py` validates assertions against stored entity facts.
**LeanKG adaptation:**
- New module: `consistency_checker` — runs on `detect_changes` or standalone
- Checks: annotations referencing deleted code, documented_by links to moved files, stale clusters
- Severity: 🔴 BROKEN, 🟡 STALE, 🟢 CURRENT
- New MCP tool: `check_consistency`, new CLI: `leankg check-consistency`
</details>
<details>
<summary>US-MP-06: Cross-Domain Tunnels</summary>
**MemPalace inspiration:** "Tunnels" auto-connect rooms from different wings when the same topic appears.
**LeanKG adaptation:**
- Auto-detect shared domain concepts across clusters
- Create `tunnel` relationship type linking related clusters
- New MCP tool: `find_tunnels`
- Enhance `orchestrate` to follow tunnels
</details>
<details>
<summary>US-MP-07: Wake-up Context Protocol</summary>
**MemPalace inspiration:** `mempalace wake-up` loads ~170 tokens of L0+L1.
**LeanKG adaptation:**
- New MCP tool: `wake_up` — returns compressed project summary (~170 tokens)
- Content: project name, languages, top directories (wings), recent hotspots, critical files
- Cached in `.leankg/wake_up.txt`, regenerated on re-index
</details>
<details>
<summary>US-MP-08: Folder Structure as Graph Edges</summary>
**MemPalace inspiration:** MemPalace's wing → room → closet → drawer is a spatial hierarchy. Each level is a navigable node with typed edges.
**LeanKG adaptation:**
- **`directory` element type** — every indexed directory becomes a first-class node
- **`contains` edges for full hierarchy:**
- `directory → directory` (e.g., `src/` contains `src/graph/`)
- `directory → file` (e.g., `src/graph/` contains `query.rs`)
- `file → function/class` (existing behavior)
- **qualified_name format:** `src/graph/` for directories (trailing slash distinguishes from files)
- **metadata on directory nodes:** `child_count`, `language_distribution`, `total_lines`
- **Impact analysis at directory level:** `get_impact_radius("src/indexer/")` shows all affected modules
- **Cluster-to-directory alignment:** When Leiden clusters map to physical directories, link them
- **Wake-up context:** L0/L1 lists top-level directories as "palace wings"
- **Folder-scoped search:** `search_code` and `query_file` accept directory qualified names
```
Palace Mapping:
Wing (project area) → src/ [directory node]
Room (module) → src/graph/ [directory node]
Closet (file) → src/graph/query.rs [file node]
Drawer (element) → query.rs::GraphEngine [function node]
All connected by `contains` edges. Traversal = BFS from any directory.
```
</details>
### 3.10 Graphify-Inspired Stories (US-GF-01 to US-GF-17)
> **Source:** Competitive analysis of [Graphify](https://github.com/Graphify-Labs/graphify) — AI coding-assistant skill that builds a queryable knowledge graph from code (tree-sitter, no LLM) plus optional docs/media. Key differentiators: `path` / `explain` / `query` agent verbs, every edge tagged `EXTRACTED|INFERRED|AMBIGUOUS`, god-node + surprising-connection reports, WHY/ADR rationale nodes, PR community conflict triage, and a work-memory reflect loop. Full matrix: [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md) (2026-07-20) + [`graphify-comparison-2026-07-13.md`](analysis/graphify-comparison-2026-07-13.md).
>
> **LeanKG keep / do not regress:** TOON/RTK token compression, requirement↔code traceability, microservice topology, severity-graded impact radius, CozoDB/RocksDB persistence, multi-project HTTP deploy.
>
> **Company rule:** Steal Graphify **packaging** (install, report, honest edges, HTML share). Do **not** chase multimodal or NetworkX. Prioritize §1.1 P1 queue for cost.
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter IDs for this section (`US-*` / related).
**Detailed Feature Descriptions:**
<details>
<summary>US-GF-01: Shortest Path</summary>
**Graphify inspiration:** `graphify path "FastAPI" "ModelField"` returns hop-by-hop edges with relation + confidence tags.
**LeanKG adaptation:**
- New MCP tool: `shortest_path(source, target, max_hops?)`
- New CLI: `leankg path <a> <b> [--max-hops N]`
- Resolve inputs by qualified_name, symbol name, or fuzzy label
- Return ordered hops: `{from, to, relation, confidence_label, source_file}`
- Prefer EXTRACTED edges when multiple equal-length paths exist
</details>
<details>
<summary>US-GF-02: Explain Node</summary>
**Graphify inspiration:** `graphify explain "APIRouter"` shows source, community, degree, and neighbor list.
**LeanKG adaptation:**
- New MCP tool: `explain_node(name_or_qn)`
- Aggregate: definition site, cluster membership, degree (in/out), top neighbors by relation type, importance/god-node rank if available
- Reuse `get_clusters`, dependents/dependencies, call graph — single agent-facing response
</details>
<details>
<summary>US-GF-03: NL Scoped Subgraph Query</summary>
**Graphify inspiration:** `graphify query "what connects auth to the database?"` returns a budgeted subgraph, not a full dump.
**LeanKG adaptation:**
- New MCP tool: `query_graph(question, token_budget?)`
- Pipeline: seed retrieval (keyword + optional embeddings) → bounded BFS/DFS expand → budget trim → TOON response
- Distinct from `orchestrate` (routing) and `kg_semantic_context` (embed pipeline): oriented to *connection* questions
- Surface confidence_label on every returned edge
- CLI: `leankg graph-query "<question>"` and `leankg query --kind subgraph "<question>"` (FR-GF-06; existing `--kind name|type|…` unchanged)
**Status (2026-07-19):** **DONE** — MCP + CLI + unit tests (`src/graph/nl_query.rs`). Release gate: `REL-042`.
</details>
<details>
<summary>US-GF-04: Edge Provenance Labels</summary>
**Graphify inspiration:** Every edge is `EXTRACTED` (explicit in source), `INFERRED` (resolver-derived), or `AMBIGUOUS` (needs review).
**LeanKG adaptation:**
- Map existing `resolution_method` (`name`, `name_file_hint`, `unresolved`, future `typed`) to provenance labels
- Store `confidence_label` on Relationship metadata; keep numeric confidence for impact severity
- Propagate labels through `get_impact_radius`, `get_call_graph`, `shortest_path`, `query_graph`, **and ui-v2 Sigma edge tooltips** (FR-GF-09)
- Prefer EXTRACTED edges when ranking equal-length paths
</details>
<details>
<summary>US-GF-05: God-Node Ranking</summary>
**Graphify inspiration:** Report highlights most-connected concepts; optional hub exclusion for utilities.
**LeanKG adaptation:**
- Precompute degree / PageRank-like importance at index time (aligns with enhancement-analysis Priority 2)
- Expose via `get_architecture` hotspots and new `get_god_nodes(limit, exclude_hubs_percentile?)`
- CLI: `leankg gods [--limit N]`
</details>
<details>
<summary>US-GF-06: GRAPH_REPORT.md</summary>
**Graphify inspiration:** Three artifacts after build: `graph.html`, `GRAPH_REPORT.md`, `graph.json`.
**LeanKG adaptation:**
- On `index` / `leankg report`: write `.leankg/GRAPH_REPORT.md` (FR-GF-13)
- Sections: god nodes, surprising cross-cluster edges, confidence distribution, 4–5 suggested agent questions
- Web UI Overview link + MCP `get_graph_report` (FR-GF-14 DONE)
- Optional MCP resources: report / god-nodes / surprises (agent-readable without inventing tool calls)
</details>
<details>
<summary>US-GF-07: Rationale / WHY Nodes</summary>
**Graphify inspiration:** `# NOTE:` / `# WHY:` / `# HACK:` and ADR/RFC citations become first-class nodes linked to code.
**LeanKG adaptation:**
- Extractor pass for comment markers + markdown ADR/RFC links
- New element type: `rationale` with `explains` relationship to code elements
- Searchable via `search_code` / `search_annotations` / `query_graph`
</details>
<details>
<summary>US-GF-08: PR Impact Dashboard</summary>
**Graphify inspiration:** `graphify prs`, `--triage`, `--conflicts` (PRs sharing communities = merge-order risk).
**LeanKG adaptation:**
- CLI: `leankg prs [number] [--triage] [--conflicts]`
- MCP: `list_prs`, `get_pr_impact`, `triage_prs`
- Combine `detect_changes` + cluster membership of touched files
- Conflicts: PRs whose changed files share clusters
</details>
<details>
<summary>US-GF-09: Work-Memory Reflect Loop</summary>
**Graphify inspiration:** `save-result` + `reflect` → `LESSONS.md` and learning overlay that biases `explain` / `query`.
**LeanKG adaptation:**
- MCP: `report_query_outcome(question, nodes[], outcome: useful|dead_end|corrected)`
- Aggregate into `.leankg/reflections/LESSONS.md`
- Optional overlay tags on nodes: preferred / tentative / contested
- Feeds context-quality loop from enhancement-analysis Priority 6
</details>
<details>
<summary>US-GF-10..12: Coverage & Portability</summary>
**US-GF-10 Languages:** Prioritize high-demand gaps (Vue/Svelte/Astro, shell, Scala) before long-tail grammars.
**US-GF-11 Portable snapshot:** Export merge-friendly `graph-snapshot.json` (relative paths); optional git merge driver. Complements RocksDB deploy — does not replace it.
**US-GF-12 SQL schema:** Optional extractor for `.sql` + `leankg extract --postgres <dsn>` creating table/FK nodes linked to ORM/repository code when detectable.
</details>
<details>
<summary>US-GF-13..17: Packaging & company adoption (v3.7.8)</summary>
**US-GF-13 HTML export (Must / P1):** As a developer, I run `leankg export html` (bounded path/community) and get a single-file graph artifact I can open offline or attach to a PR — without replacing the live ui-v2 explorer.
**US-GF-14 Three-verb narrative (Must / P1):** As an agent user, README / AGENTS.md / skills lead with **path · explain · query** before the full MCP catalog, so cheap connection questions do not trigger grep/cat dumps.
**US-GF-15 Install matrix (Should / P2):** As a platform eng, `leankg install` covers Cursor, Claude Code, Codex, OpenCode, Gemini (+ more over time) with graph-first guidance.
**US-GF-16 Reflect skill (Should / P2):** As an agent, default skill guidance calls `report_query_outcome` after useful/dead_end answers and loads LESSONS at session start.
**US-GF-17 Always-on graph-first (Must / P1):** As a company, `leankg install` writes always-apply Cursor rules / Claude PreToolUse (or equivalent) that **nudge** (optional **strict**: block first raw Read) agents to query LeanKG before grep — the primary lever for §1.1 token savings.
</details>
### 3.11 CBM Structural Parity Stories (US-CBM) — merged from `prd-structural-parity-cbm.md`
> **Source:** Competitive analysis of [codebase-memory-mcp (CBM)](https://github.com/DeusData/codebase-memory-mcp) v0.9.0 vs LeanKG (current: v0.17.9). Deep comparison notes also in `docs/analysis/` historical stubs.
>
> **Product rule:** Lean into business-context depth (ontology, knowledge, env, Android, req↔code). Close structural gaps that erode agent trust. Do **not** chase Pure-C / 158-language parity.
>
> **Tracks:** A Activate · B Structural · C Platform · D Dual-run escape hatch · E 3D graph UI
>
> **Codebase status audit:** 2026-07-14 against `src/` (v0.17.9)
#### Positioning (summary)
| Dimension | LeanKG | CBM |
|-----------|--------|-----|
| Stack | Rust + CozoDB/RocksDB | Pure C + SQLite |
| MCP | 85 tools (current), stdio + HTTP/SSE + REST | ~14 tools, stdio |
| Strength | Ontology, knowledge, env/incidents, Android, Docker+RocksDB, RTK | Speed, 158 langs, Hybrid LSP, clones, CROSS_*, static binary |
| Call resolve today | `name` / `name_file_hint` / `unresolved` + confidence | Hybrid LSP Tier 1/2/3 |
#### User stories — Track A Activate
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter IDs for this section (`US-*` / related).
#### User stories — Track B Structural
#### User stories — Track C Platform
#### User stories — Track D Dual-run
#### User stories — Track E 3D Visualization
<details>
<summary>US-CBM detailed notes (implementation evidence)</summary>
**DONE evidence (post-v3.6 audit, 2026-07-14):**
- `get_architecture` / `get_graph_schema` / `find_dead_code` — `src/mcp/tools.rs`, `src/mcp/handler.rs`, `src/graph/query.rs`
- `resolution_method` + `confidence` — `src/indexer/call_graph.rs`
- Routes + `http_calls` — `src/indexer/route_extractor.rs`, `RelationshipType::HttpCalls` in `src/db/models.rs`
- `wake_up` — `src/mcp/handler.rs` (also closes MemPalace US-MP-07)
- `LSP` module — `src/lsp/{bridge,client,config,mod}.rs`; `resolve_with_lsp` MCP; `leankg lsp-resolve` CLI; `IndexerConfig::typed_resolve` (`8971dc5`)
- Event edges `emits` / `listens_on` — `src/db/models.rs` + `25a3b37`
- Clones — historically `find_clones` MCP + `leankg clones` + `SimilarTo` (`55e6e72`); **MCP/CLI hard-removed 2026-07-20** (`SimilarTo` rel-type stub may remain in `models.rs`)
- Cross-repo similar — `find_cross_repo_similar` MCP + `CrossRepoSimilar` (`ab16c9b`)
- Hot-path cache — `src/cache/hot_path.rs` + `836f0a3`
- Temporal graph fields — `src/db/models.rs` `valid_from` / `valid_to` (`bc9cc53`)
- Consistency checker — `check_consistency` MCP + `leankg check-consistency` (`60a6111`)
- Tunnels — `find_tunnels` MCP + `leankg tunnels` + `Tunnel` (`5b6547e`)
- Agent personas — `agent_focus` + `agent_diary_{read,write}` (`1ea4bcd`)
- Rationale extraction — `src/indexer/rationale_extractor.rs` (`b0c9477`)
- PR impact dashboard — `get_pr_impact` MCP + `leankg prs` (`30e41f0`)
- Work-memory reflect — `report_query_outcome` + `Lessons` aggregation (`373e808`)
- Portable snapshot — `export_graph_snapshot` MCP (`0087991`)
- Team / on-call — `get_team_map` MCP (`3368b5f`)
- Cluster SKILL — `get_cluster_skill` MCP (`10b15a0`)
- Overview context — `get_overview_context` MCP (`9124959`)
- CI/CD auto-update — `.github/workflows/leankg-graph-update.yml` (`eb3d331`)
- Vue + Svelte — `src/indexer/sfc.rs` (regex; **not called from index walk**) (`e617a49`)
- SQL DDL — `src/indexer/sql.rs` (**not called from index walk**) (`de314eb`)
- Swift — `src/indexer/swift.rs` (**not called from index walk**) (`7027d6b`)
**PENDING evidence:**
- No `typed` `resolution_method` produced at index time; LSP bridge returns `LspLocation[]` but does not yet write CALLS edges with `resolution_method=typed`
- No `graph-ui/` directory; no `get_graph_layout` / 3D scene
- No formal `resources/read` endpoint for `get_overview_context` (tool-only)
- Swift / Vue / Svelte / SQL extractors exist as modules but `.swift` / `.vue` / `.svelte` / `.sql` are absent from `find_files_sync`
**Won’t Have (this program):** Full 158-language parity; Pure-C rewrite; replace Cozo/RocksDB; full Hybrid LSP for all CBM families in one release; drop HTTP/SSE/REST or Docker team path; **custom MinHash/LSH or Cozo `::lsh` clone ANN** (v3.6.2 — semantic HNSW only).
</details>
### 3.12 Team Knowledge Infrastructure (US-V2) — merged from `prd-leankg.md` v2
> **Vision:** Evolve from local-first single-dev tool to shared knowledge backbone for multi-service teams: environment-scoped graph, incident knowledge, CI freshness, token-budgeted MCP tools.
>
> **Codebase status audit:** 2026-07-14 (v0.17.9)
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter IDs for this section (`US-*` / related).
**v2 success metrics (targets):**
| Metric | Target |
|--------|--------|
| Full service context in Cursor | < 5s |
| Context tokens per session (LeanKG queries) | < 2,000 |
| Graph freshness after release hook | < 3 minutes |
| Env conflict catch rate (schema) | > 80% |
<details>
<summary>US-V2 data model (from HLD/PRD v2)</summary>
**Incident node fields:** id, env, title, severity (P0–P3), occurred_at, resolved_at, root_cause, resolution, affected services, trigger_pattern, prevention, tags, author, linked_ticket
**v2 edges:** `caused_incident`, `resolved_by`, `conflicts_with`, `deployed_to`, `supercedes`
**Service metadata:** version, deploy_env, slo_p99_ms, health_endpoint, on_call, incident_count, last_incident, tags
</details>
### 3.13 Optimized Local-First Vector Graph Engine (US-VE) — v3.7.0
> **Former P0 — COMPLETE on PR #80.** Current highest focus is **§3.15 Day-2 Embed Resume**. Core implementation **merged to `origin/main`** (`dbc22c4` / #79). Full ordered queue: [`prd-task-tracker.md`](prd-task-tracker.md).
>
> **Epic:** Replace mmap-heavy / opaque vector I/O with a **3-tier LocalEngine** (and CloudEngine twin) so semantic+LSP retrieval stays surgical under M2 Pro / 16GB / 256GB SSD, and scales to Linux x86_64 + TiKV without rewriting MCP/CLI callers.
>
> **Depends on:** FR-HNSW-* product path (semantic ANN UX). **Does not cancel** FR-HNSW-B until LocalEngine recall/latency gates pass and factory switch is default for Local mode.
>
> **P0 gate complete on `feature/vector-engine-gate`:** US-VE-* + FR-VE-* quality gates **DONE**. `evaluate_gate` sets `ready_for_default` under `LEANKG_VE_GATE_FULL=1`. Cozo remains runtime default until callers honor `preferred_ann_backend`.
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter Focus=`P0` / `US-VE-*` / `FR-VE-*`.
**Acceptance criteria (epic-level):**
- **Given** a warm LocalEngine index of ≥1M SQ8 chunks on reference M2 Pro, **When** an agent issues semantic retrieval, **Then** P95 end-to-end time-to-context JSON is < 100ms and ANN-only P95 is < 50ms.
- **Given** a 2GB cgroup limit, **When** the engine auto-tunes RocksDB block cache + rayon threads, **Then** the process is never OOM-killed during index+query smoke.
- **Given** Flat File append succeeds and process crashes before RocksDB offset commit, **When** the engine recovers, **Then** no dangling pointers remain and queries skip incomplete records.
- **Given** `LEANKG_VECTOR_ENGINE=local|cloud` (or equivalent), **When** the factory constructs storage, **Then** the correct backend enum variant is used (unit-asserted).
### 3.14 Semantic MCP Agent UX Enhancements (US-SEM) — v3.7.1
> **Trigger:** Live Docker MCP probe 2026-07-17 ([`docs/semantic-search-mcp-verification-2026-07-17.md`](semantic-search-mcp-verification-2026-07-17.md)). Pipeline (HNSW ANN → cross-encoder rerank → optional graph hop) is **correct**; these stories improve agent-facing honesty, budgets, transport resilience, and release smoke — **not** recall correctness.
>
> **Depends on:** FR-HNSW-D / FR-V2-06 / FR-V2-07 (shipped). **Does not** reopen Cozo vs LocalEngine cutover (FR-VE-GATE).
>
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-SEM-*` / `FR-SEM-*` / `REL-051` / `REL-054`. **P0 open:** `US-SEM-06` / `FR-SEM-07` / `REL-054`.
#### US-SEM-01 — Honest token accounting on truncated MCP payloads (Should Have)
**As an** AI agent, **I want** both *delivered* and *pre-truncation* token costs when a tool response is truncated, **so that** I can budget context windows without underestimating cost by 3–4×.
**Acceptance criteria:**
- **Given** a tool response with `_token_budget.truncated: true`, **When** the agent reads the envelope, **Then** both delivered token count and `_token_budget.actual` (pre-trim) are present and unambiguous.
- **Given** `truncated: false`, **When** the agent reads the envelope, **Then** delivered ≈ actual (within rounding).
- **Given** docs / MCP skill guidance, **When** `truncated: true`, **Then** guidance tells agents to budget **≥ 3×** the delivered figure (or use `actual`).
#### US-SEM-02 — Adequate budgets for ontology-heavy tools (Should Have)
**As an** AI agent using `concept_search` / `kg_semantic_context`, **I want** tool-specific token maxima (not the 1000 default), **so that** ontology + graph payloads are not silently cut mid-result.
**Acceptance criteria:**
- **Given** a populated ontology + index (verification setup), **When** `concept_search` / `kg_semantic_context` run typical probes, **Then** default maxima are ≥ sibling `kg_*` tools (2k–4k range) unless the caller overrides.
- **Given** a response that still truncates, **When** `_token_budget` is inspected, **Then** `max` reflects the tool-specific budget (not anonymous default).
#### US-SEM-03 — Resilient MCP HTTP for long semantic calls (Should Have)
**As an** AI agent calling `kg_semantic_context` (or similar) over HTTP/SSE, **I want** transient socket drops mitigated (retry-safe server + documented client retry), **so that** one flake does not fail a session.
**Acceptance criteria:**
- **Given** a transient “socket connection was closed unexpectedly” on a read-only semantic tool, **When** the same args are retried once, **Then** the call succeeds without corrupt state (matches 2026-07-17 probe).
- **Given** long-lived `:9699` daemons, **When** operators follow doctor / restart guidance, **Then** docs link [`docs/analysis/mcp-http-stability-analysis-2026-05-05.md`](analysis/mcp-http-stability-analysis-2026-05-05.md) and keep stale-listener cleanup as the ops path.
#### US-SEM-04 — Semantic hit diversity across files (Could Have)
**As an** AI agent, **I want** top-k `semantic_search` hits to diversify across files when ANN+rerank collapses to one module, **so that** I see alternate entry points without a second query.
**Acceptance criteria:**
- **Given** a query whose top-10 would otherwise be ≥70% one file (as in the MCP-dispatch probe), **When** diversity mode is on (default or flag), **Then** at least N distinct `file_path` values appear in top-k (N configurable; default ≥3 for k=10).
- **Given** diversity mode off, **When** the same query runs, **Then** ranking matches current HNSW+rerank order (no regression).
#### US-SEM-05 — Exclude UI-bundle / benchmark noise from semantic seeds (Must Have)
**As an** AI agent, **I want** `semantic_search` to ignore minified embedded UI assets and (by default) benchmark helpers, **so that** HNSW / scoring queries return readable Rust retrieval symbols instead of single-letter JS or `verdict` helpers.
**Acceptance criteria:**
- **Given** Probe G query (`HNSW approximate nearest neighbor vector index`), **When** `semantic_search` runs, **Then** top-k does **not** include `src/embed/assets/*.js` (or any `embed/assets/` path).
- **Given** Probe H query (`vector similarity scoring`) without the word "benchmark", **When** `semantic_search` runs, **Then** `src/benchmark/**` candidates are filtered before rerank.
- **Given** a query containing `"benchmark"`, **When** `semantic_search` runs, **Then** `src/benchmark/**` may appear (gated keep).
- **Evidence baseline:** [`docs/semantic-search-mcp-verification-2026-07-18.md`](semantic-search-mcp-verification-2026-07-18.md) Probes G/H.
#### US-SEM-06 — Mega-graph HNSW semantic search must not OOM MCP (**P0** / Must Have)
**As an** AI agent on a mega-graph Docker MCP mount (~600k+ elements, ~150k vectors), **I want** `semantic_search` and `kg_semantic_context` to complete under the documented MCP memory budget, **so that** natural-language retrieval does not kill `:9699` and strand the rest of the tool surface.
**Acceptance criteria:**
- **Given** an indexed mega project (`project=/workspace-other` or equivalent container mount) with an existing embedding index, **When** `semantic_search(query=…, limit≤5)` runs, **Then** the call returns `method: hnsw+rerank` (or documented degrade) **without** container `OOMKilled`, process crash, or HTTP disconnect; `/health` stays ok during and after the call.
- **Given** the same setup, **When** `kg_semantic_context` runs a short NL query, **Then** the same stability ACs hold (no mega `all_elements()` dump).
- **Given** logs during the call, **When** inspected, **Then** there is **no** unbounded `all_elements()` / full-graph element cache load for seed hydration (ANN candidates + paginated / keyed element fetch only).
- **Given** the same binary on a small project (`/workspace`), **When** `semantic_search` runs, **Then** existing HNSW+rerank behavior remains GREEN (no regression).
- **Evidence of failure (baseline):** [`docs/reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md`](reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md) — mega FAIL / small PASS; mem peak ~3.3 GiB before kill.
**Temporary agent workaround (not the fix):** on mega-graphs prefer `concept_search` → `search_code` / `find_function` until FR-SEM-07 lands.
#### US-MG-TOOL-01 — Mega-safe prefer-order + NL tools (**P0** / Must Have)
**As an** AI agent on a mega-graph mount, **I want** `concept_search` and `query_graph` to complete without dumping the full element/relationship tables, **so that** prefer-order discovery and NL subgraph stay usable under the documented MCP memory budget.
**Acceptance criteria:**
- **Given** mega `/workspace-other`, **When** `concept_search(query="authentication")` runs, **Then** it returns within seconds without MCP restart; no full-table `load_indexed_code_elements` / equivalent.
- **Given** mega, **When** `query_graph(question="what connects auth to payment")` runs, **Then** it completes without `all_elements()` / `all_relationships()` WARN on that path; response respects `token_budget`.
- **Given** mega, **When** `get_clusters` runs and live Louvain is refused, **Then** if `cluster_id` is populated it returns precomputed clusters; else a structured empty + offline-assign hint (not a hang).
- **Evidence of failure (baseline):** [`docs/reports/ce03fd8-docker-mcp-full-tool-test-2026-07-20.md`](reports/ce03fd8-docker-mcp-full-tool-test-2026-07-20.md).
### 3.15 Day-2 Embed Resume (US-EMBED) — v3.7.2 — **DONE (prior P0)**
> **Trigger:** Turning embedding on (standalone Docker `embed --wait`, or Docker MCP with `LEANKG_EMBED_BACKGROUND` / boot / setup) against a persisted RocksDB volume looks like a full cold rebuild — unacceptable CPU/RAM/time on mega-graphs.
>
> **Universal rule:** **Resume if embed data exists; cold/fresh only if none exists.** Same rule for every entry path.
>
> **Depends on:** `embedding_state` + incremental filter (FR-HNSW-E foundation). **Closes the gap** so day-2 behavior matches the product promise in v3.6.3 (“day-2 incremental embed stays fast”).
>
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter Focus=`P0` / `US-EMBED-*` / `FR-EMBED-RESUME-*` / `REL-052`.
#### US-EMBED-01 — Second standalone embed updates deltas only (Must Have)
**As an** operator embedding a large mounted project via Docker `embed --wait`, **I want** a second run on unchanged code to skip already-`fresh` vectors, **so that** I do not burn another full cold-embed (hours / multi-GB) of CPU and RAM.
**Acceptance criteria:**
- **Given** a RocksDB project with `embedding_state=fresh` for (nearly) all targeted types after a successful first embed, **When** the same `embed --wait --types function,method` command runs again with no code changes, **Then** ONNX work embeds ≈0 new vectors (`skipped_fresh` ≈ work list) and wall time is **minutes not hours** on mega-graphs.
- **Given** a small set of new/changed functions, **When** day-2 embed runs, **Then** only those dirty QNs are re-inferred; other fresh rows stay untouched.
- **Given** docs / operator scripts, **When** examples show standalone Docker embed, **Then** they use container path placeholders (`/workspace-other`) and document that the named RocksDB volume must be reused.
#### US-EMBED-02 — Interrupted embed resumes without discarding fresh work (Must Have)
**As an** operator whose embed job is killed (OOM, Ctrl-C, container stop), **I want** the next embed (CLI or Docker MCP) to resume from persisted `embedding_state`, **so that** already-written fresh vectors are not recomputed.
**Acceptance criteria:**
- **Given** an embed interrupted after N batches committed `fresh`, **When** embed is started again (same project / same RocksDB), **Then** those N QNs are skipped and only remaining dirty/missing QNs are processed.
- **Given** a stale `embed.lock` with no live PID, **When** a new embed starts, **Then** the lock is cleared or stolen safely (existing lock semantics) without wiping vectors.
#### US-EMBED-03 — Zero-dirty run does not tear down HNSW (Must Have)
**As an** operator, **I want** a no-op day-2 embed to leave the HNSW index intact, **so that** semantic search stays available and the process does not pay O(N log N) rebuild cost for zero writes.
**Acceptance criteria:**
- **Given** `to_embed` is empty and orphan reap is empty, **When** any embed path runs, **Then** the process does **not** drop or recreate `embedding_vectors:vec_idx`.
- **Given** `to_embed` is non-empty, **When** bulk write path runs, **Then** existing drop-before-bulk / recreate-after (or equivalent correct incremental update) remains allowed.
#### US-EMBED-04 — Docker MCP / boot embed resumes existing data (Must Have)
**As an** operator who turns embedding on inside Docker LeanKG (e.g. `LEANKG_EMBED_BACKGROUND=1`, `LEANKG_EMBED_ON_BOOT=1`, `LEANKG_DOCKER_SETUP=1`, or `docker-up.sh`), **I want** the service to detect existing vectors/`embedding_state` on the RocksDB volume and **resume**, **so that** enabling embed after a prior cold fill (or after MCP-only restarts) never wipes or re-embeds the whole graph.
**Acceptance criteria:**
- **Given** a named RocksDB volume already containing embeddings for `/workspace` or `/workspace-other`, **When** the container starts with any embed-on path enabled, **Then** LeanKG runs **incremental resume** (skip `fresh`); it does **not** treat the start as a cold wipe.
- **Given** a project with **no** embedding data (empty state / no vectors), **When** embed is turned on, **Then** a **cold/fresh** fill is allowed and expected.
- **Given** embed remains off (`LEANKG_EMBED_BACKGROUND=0`, `LEANKG_EMBED_ON_BOOT=0`), **When** MCP restarts, **Then** existing embed data is left intact and semantic tools use whatever HNSW is already present.
- **Given** docs (`entrypoint.sh`, compose, AGENTS), **When** operators enable Docker embed, **Then** docs state the resume-vs-cold rule explicitly.
#### US-EMBED-05 — MCP idle-gated partial embed toggle (Must Have)
**As an** operator with `LEANKG_EMBED_ON_BOOT=0` / `LEANKG_EMBED_BACKGROUND=0`, **I want** MCP `embed_control` to arm/disarm in-process embedding only when the server is idle and under an RSS fraction of the container budget, **so that** day-2 resume does not starve search or rebuild cold when vectors already exist.
**Acceptance criteria:**
- **Given** MCP is healthy and boot embed is off, **When** `embed_control(action=on)`, **Then** the job waits for idle (`LEANKG_EMBED_IDLE_AFTER_SECS`) + RSS headroom, then runs **Incremental** resume by default (`full=false`).
- **Given** existing vectors and an unchanged graph, **When** the job runs, **Then** status reports `mode=incremental`, high `skipped_fresh`, `embedded≈0`, and the process does **not** load ONNX or drop HNSW.
- **Given** a running/armed job, **When** `embed_control(action=off)`, **Then** the in-process worker cooperatively cancels (no `SIGTERM` to Docker PID 1).
- **Given** `mode=partial` (default), **When** embed runs under MCP, **Then** it duty-cycles batches, yields on MCP activity (`paused_yield`), and caps soft RSS to `LEANKG_EMBED_RSS_FRACTION` of the container budget.
---
### 3.16 MCP Tool Surface Rationalization (US-SURF) — v3.7.4
> **Trigger:** Agent-facing overlap among discovery / semantic / identity tools plus legacy `mcp_*` names. Review evidence: live MCP schemas + `src/mcp/handler.rs` / `tools.rs`; redundancy matrix in `tests/redundant_tools_matrix.rs`.
>
> **Baseline:** `ToolRegistry::list_tools()` ≈ **85** tools (7 `mcp_*`). Soft-deprecations do not shrink `tools/list` until removed.
>
> **Policy:** Document first → hard-delete true subsets → soft-deprecate with explicit replacements. Prefer wrong-tool prevention over cosmetic renames.
>
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-SURF-*` / `FR-SURF-*` / `REL-053`.
#### Prefer-order (canonical)
**Search triple (discovery):**
1. `concept_search` — domain terms / aliases / matched concepts payload
2. `semantic_search` — natural language; HNSW+rerank when embeddings exist, else ontology-first `safe_discover`
3. `search_code` — exact/name + `element_type` filter; ontology-first on mega-graphs
**Semantic triple (context):**
1. `semantic_search` — discovery page (ANN if available, else ontology+name)
2. `kg_semantic_context` — ranked seeds + 1–2 hop neighborhood (**embeddings required**)
3. `kg_context` — ontology match + expand (**no vectors**)
**Identity triple (session start):**
1. `get_overview_context` — structured L0 + L1 + wake summary
2. `load_layer` — progressive L0 / L1 / L2 / L3 chooser
3. `get_architecture` — deep single-call overview
4. `wake_up` — **deprecated path** (cached L0+L1 text in `.leankg/wake_up.txt`)
#### US-SURF-01 — Agents know which search/semantic tool to call first (Must Have)
**As an** AI agent, **I want** each overlapping search/semantic tool's schema to state when to use it and what to try next, **so that** I do not burn retries on the wrong surface.
**Acceptance criteria:**
- **Given** `tools/list`, **When** an agent reads `semantic_search`, **Then** the description states the dual path (HNSW+rerank if embeddings exist; else ontology-first) — not “ANN-only”.
- **Given** `search_code` / `concept_search` / `semantic_search` / `kg_semantic_context` / `kg_context`, **When** schemas are listed, **Then** each has a one-line prefer-order hint consistent with §3.16.
- **Given** docs / AGENTS / LeanKG skill, **When** search guidance is updated, **Then** it matches the same prefer-order (no conflicting “always grep / always semantic” advice).
#### US-SURF-02 — Remove zero-value / superseded MCP tools (Must Have)
**As an** AI agent, **I want** dead or strict-subset tools removed from the registry, **so that** `tools/list` is smaller and I cannot call obsolete names by mistake.
**Acceptance criteria:**
- **Given** a build after this story, **When** `ToolRegistry::list_tools()` runs, **Then** `mcp_hello`, `mcp_impact`, and `get_doc_for_file` are **absent**.
- **Given** callers needing impact analysis, **When** they use `get_impact_radius`, **Then** they retain severity / confidence / `project=` / `compress_response` (strict superset of former `mcp_impact`).
- **Given** callers needing docs for a file, **When** they use `find_related_docs`, **Then** they get `documented_by` **and** `references` (superset of former `get_doc_for_file`).
- **Given** `tests/redundant_tools_matrix.rs`, **When** the suite runs, **Then** it reflects the removals (no stale SUPERSEDED rows for deleted tools).
#### US-SURF-03 — Soft-deprecate wake_up in favor of get_overview_context (Should Have)
**As an** AI agent starting a session, **I want** one structured overview tool, **so that** I do not choose among three identity loaders incorrectly.
**Acceptance criteria:**
- **Given** `wake_up` still registered during the deprecation window, **When** its description is read, **Then** it is marked deprecated and points to `get_overview_context` (and optionally `load_layer` for progressive budgets).
- **Given** a caller that previously used `wake_up` for L0+L1, **When** migrated, **Then** they do **not** replace it with `load_layer(layer="L0")` alone (L0 is identity-only ~50 tok; wake_up is L0+L1 ~170 tok).
- **Given** one release after soft-deprecation lands, **When** product decides, **Then** hard removal is allowed if call telemetry / docs are clear.
#### US-SURF-04 — Soft-deprecate search_by_environment (Should Have)
**As an** AI agent, **I want** environment scoping via the `env=` parameter on primary search/kg tools, **so that** I do not maintain a parallel env-only search tool.
**Acceptance criteria:**
- **Given** `search_by_environment` during deprecation, **When** its schema is read, **Then** it points callers to `env=` on `search_code` / `semantic_search` / `concept_search` / `kg_*`.
- **Given** primary search tools, **When** `env` is set, **Then** behavior remains unchanged (no regression).
#### US-SURF-05 — Optional unify doc structure tools (Could Have)
**As an** AI agent exploring documentation, **I want** one doc-structure tool with a format flag, **so that** I do not pick between `get_doc_tree` and `get_doc_structure`.
**Acceptance criteria:**
- **Given** mega-graphs, **When** either tool (or the merged tool) runs, **Then** it must not call unbounded `all_elements()` (refuse or paginate — safety before merge).
- **Given** a merge, **When** `format` is `"tree"` | `"list"`, **Then** payloads match today's tree vs flat list shapes.
- **Out of scope for Must Have:** cosmetic merge alone without mega-graph safety.
## 4. Implementation Status Summary
> **Implementation status:** see [`prd-task-tracker.md`](prd-task-tracker.md) — Summary counts + Active session (open work) + Master table.
## 5. Functional Requirements
### 5.1 Core Features (DONE)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.2 GitNexus Enhancements (DONE)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.3 AB Testing & Validation (DONE)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.4 RTK Compression (DONE)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.5 Infrastructure Features (DONE)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.6 MemPalace-Inspired Features
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.7 Massive Graph UI (DONE)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.8 Multi-Language Support
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
> **Indexed vs module-only (audit 2026-07-15):** “DONE (indexed)” means the extension is in `find_files_sync` and extraction runs on index. “PARTIAL (unwired)” means an extractor module/tests exist but the extension is **not** scanned.
| Language | Extensions | Extractor Status | Parser / module |
|----------|-----------|-----------------|-----------------|
| Go | `.go` | DONE (indexed) | tree-sitter-go |
| TypeScript/JavaScript | `.ts`, `.tsx`, `.js`, `.jsx` | DONE (indexed) | tree-sitter-typescript |
| Python | `.py` | DONE (indexed) | tree-sitter-python |
| Rust | `.rs` | DONE (indexed) | tree-sitter-rust |
| Java | `.java` | DONE (indexed) | tree-sitter-java |
| Kotlin | `.kt`, `.kts` | DONE (indexed) + Android depth | tree-sitter-kotlin-ng |
| Dart | `.dart` | DONE (indexed) (`7ec6484`) | tree-sitter-dart |
| XML | `.xml` | DONE (indexed) (`92db9aa`) + Android | tree-sitter-xml / Android extractors |
| Terraform | `.tf` | DONE (indexed) | Custom extractor |
| CI/CD YAML | `.yml`, `.yaml` | DONE (indexed) | GitHub Actions, GitLab CI, Azure Pipelines |
| Markdown | `.md` | DONE (doc indexer) | pulldown-cmark |
| Swift | `.swift` | PARTIAL (unwired) — `src/indexer/swift.rs` (`7027d6b`) | regex stub |
| Vue (SFC) | `.vue` | PARTIAL (unwired) — `src/indexer/sfc.rs` (`e617a49`) | regex stub |
| Svelte (SFC) | `.svelte` | PARTIAL (unwired) — `src/indexer/sfc.rs` (`e617a49`) | regex stub |
| SQL DDL | `.sql` | PARTIAL (unwired) — `src/indexer/sql.rs` (`de314eb`) | regex stub |
| C/C++ | `.cpp`, `.cxx`, `.cc`, `.hpp`, `.h`, `.c` | PARTIAL — tree-sitter parser present; **not** in current `find_files_sync` extensions list | tree-sitter-cpp |
| C# | `.cs` | PARTIAL — parser present; **not** in current index walk | tree-sitter-c-sharp |
| Ruby | `.rb` | PARTIAL — parser present; **not** in current index walk | tree-sitter-ruby |
| PHP | `.php` | PARTIAL — parser present; **not** in current index walk | tree-sitter-php |
### 5.9 Graphify-Inspired Features
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-GF-*`.
> Evidence: [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md). Deploy parity with Graphify HTTP MCP is **not** a gap — LeanKG RocksDB multi-project compose is competitive. Focus requirements on **agent query UX, edge honesty, report/HTML artifacts, always-on install**.
> **Promote to Focus P1:** FR-GF-07, FR-GF-08, FR-GF-09, FR-GF-13, FR-GF-21, FR-GF-22, FR-GF-24 (see §1.1 queue).
New FRs (v3.7.8): FR-GF-21..24 — see §5.20 table (shared with cost track for tracker visibility).
### 5.10 CBM Structural Parity Requirements (merged)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
> Canonical FR IDs retained from `prd-structural-parity-cbm.md` (FR-A/B/C/D/E). Status audited 2026-07-14 (v0.17.9).
Tracks A–E (activate / structural / platform / dual-run / 3D UI): see tracker `FR-A*` / `FR-B*` / `FR-C*` / `FR-D*` / `FR-E*`.
### 5.11 Team Infrastructure / v2 Requirements (merged from `prd-leankg.md`)
> **FRs + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-*` for this section.
### 5.12 Semantic ANN — CozoDB HNSW expansion (v3.6.2) + embed runtime (v3.6.3)
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-HNSW-*`, `FR-EMBED-*`, `FR-LSH-*`, `FR-BENCH-HNSW`.
> **Product bet:** LeanKG's strong path is **semantic search** via dense embeddings + CozoDB native `::hnsw`. Do **not** reimplement MinHash/LSH in-process, and do **not** wire Cozo `::lsh` for clones. Pattern already proven by embeddings: LeanKG builds text blobs → Cozo stores vectors + HNSW index.
>
> **Cold-embed reality (v3.6.3):** on mega-graphs, wall time is dominated by **ONNX embedding inference** (~170 vec/sec e2e → ~36 min for ~371k functions). Cozo/Redis writer-only paths are ~100k+ vec/sec. Do **not** treat storage migration (WAL-off, Redis, FalkorDB) as the primary cold-SLA lever.
**Policy (details + status in tracker):**
- Remove custom MinHash/LSH; keep Cozo `::hnsw` as shipped default until FR-VE-GATE
- MCP must not block on cold embed; **day-2 incremental / resume is P0** (FR-EMBED-RESUME-* closes gaps vs FR-HNSW-E PARTIAL)
- **Won't Do:** Cozo `::lsh` for clones; migrate KG to FalkorDB/Redis to fix cold embed
### 5.13 LSP Adoption Track from CBM (moved from former 5.12; deep compare 2026-07-15)
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-LSP-A`..`FR-LSP-D`.
> LSP-only FRs retained from the CBM deep read. Clone/LSH FRs cancelled in Section 5.12.
>
> **Intent:** close the zero-setup gap (LeanKG currently requires user-configured LSP servers) via prefab `lsp:` block, optional in-process native resolver, indexer wiring for `resolution_method=typed`, and cross-file type registry.
| ID | Priority | Requirement |
|----|----------|-------------|
| FR-LSP-A | Must Have | LeanKG-native Hybrid LSP tier — **in-process, no-spawn** type resolver for **Go + TypeScript MVP** (Python/Rust later). Runs during index; never forks gopls/tsserver. |
| FR-LSP-B | Must Have | Prefab `lsp:` block — `leankg init --with-lsp` writes default servers from the catalog (`gopls`, `typescript-language-server`, `pyright`, …) and sets `indexer.typed_resolve: go,ts`. Empty yaml still falls back to catalog at resolve time (REL-039). |
| FR-LSP-C | Must Have | Wire hybrid/LSP results into the indexer — when `typed_resolve=go,ts` (or `all`), CALLS edges get `resolution_method=typed` + high confidence before DB insert. |
| FR-LSP-D | Must Have | Cross-file type registry shared across files in the same project (functions/methods/types keyed by name, module/dir, and type+method). |
| REL-039 | Must Have | Default LSP server bootstrap fanout (FR-LSP-B) — catalog-backed prefab for gopls + tsserver + pyright (+ other catalogued languages). |
**Acceptance (MVP):**
- Given a small Go package with cross-file calls and `typed_resolve=go,ts`, indexing produces ≥1 CALLS edge with `resolution_method=typed`.
- Given a TS module pair with an exported function call, same.
- `leankg init --with-lsp` writes a non-empty `lsp.servers` map.
- Hybrid path never spawns a child process (unit-tested).
- External `resolve_with_lsp` still works when binaries are installed (existing bridge).
### 5.14 Optimized Local-First Vector Graph Engine (v3.7.0)
> **Former P0 — COMPLETE on PR #80.** Current highest focus is **§5.16 Day-2 Embed Resume**.
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-VE-*` / `US-VE-*` / Kind=`Release` (§8.4).
> **Goal:** Ultra-lightweight vector/graph storage + retrieval that works under Apple M2 Pro / 16GB / 256GB SSD and scales to Linux x86_64 + TiKV **without rewriting** MCP/CLI semantic APIs.
>
> **Coexistence:** FR-VE-GATE is **met** on PR #80 when `LEANKG_VE_GATE_FULL=1` (`ready_for_default=true`). Runtime **shipped default ANN** remains Cozo `::hnsw` (FR-HNSW-B) until callers honor `preferred_ann_backend()`. LocalEngine / CloudEngine stay opt-in via `LEANKG_VECTOR_ENGINE=local|cloud`.
>
> **Verification (2026-07-17 — PR #80):**
> - Unit: `cargo test --release --lib -- vector_engine` → **56 passed** (3 ignored full-scale)
> - CI path: `cargo test --lib` (debug) → **651 passed** after RSS `delta_ok` + i8 overflow fixes
> - E2E: `cargo test --release --test vector_engine_e2e` → **6 passed**; `LEANKG_VE_GATE_FULL=1` ignored gate → `ready_for_default=true`
> - Bench: `cargo bench --bench vector_engine_ab` → [`docs/benchmarks/vector_engine_gate_results.json`](benchmarks/vector_engine_gate_results.json)
> - A/B measured: token −**65.0%**, tool −**84.6%**, speedup **2.50×** (100 tasks; floors 60%/80%/2×)
> - Docs: product README polish (`85c1632`); semantic backlog v3.7.1 remains P2/P3
> **Hardware envelope:** Local survival cap **2GB** (Docker/cgroup) → Cloud **50–80%** of available RAM. Prefer sequential append I/O; minimize random SSD writes.
#### 5.14.1 Decoupled 3-tier storage
- Tier 1: graph topology in RocksDB (Local) / TiKV (Cloud) — metadata, AST refs, HNSW adjacency; Local RocksDB: mmap off, pin L0 filter/index, BinaryAndHash, Zstd
- Tier 2: SQ8/INT8 vectors 100% in RAM for SIMD ANN (no disk I/O on inner loop)
- Tier 3: flat binary FP32 + source payload — read once at post-filter
- Abstraction: Rust traits + static enum dispatch (`LocalEngine` | `CloudEngine`)
#### 5.14.2 Dynamic runtime adaptation
- Runtime SIMD dispatch (AVX-512 / AVX2 / NEON / scalar) — never SIGILL
- Auto-tune RocksDB block cache from cgroups / sysinfo
- Dynamic rayon pool (leave 2 cores free Local; full machine Cloud)
- HNSW M ∈ [12, 16]; raise efConstruction; recall > 90% at efSearch=50
#### 5.14.3 Flat file consistency & GC
- Dual-write order: Append → fsync → commit offsets → update RAM SQ8
- Crash recovery must leave no dangling pointers
- Zero-downtime GC (shadow paging + micro-lock delta) when fragmentation > 30%
#### 5.14.4 Tests & benches (mandatory before default switch)
Agent A/B floors (also in NFR / tracker `FR-VE-BENCH-*`):
| Metric | Target |
|--------|--------|
| Token consumption | ≥ **60%** reduction vs grep/cat baseline (stretch **61%**) |
| Tool-call frequency | ≥ **80%** reduction (stretch **84%**, aim 1-hop context) |
| Time-to-resolution | ≥ **2×** faster |
| Task success rate | ≥ baseline |
**Won't Do (this track):**
- Reopen Redis/FalkorDB as cold-embed write accelerator (still Won't Do per v3.6.3)
- Require Cloud SaaS hosting (self-hosted TiKV/CloudEngine only)
- Rewrite MCP tool names/APIs for the engine swap
### 5.15 Semantic MCP Agent UX Enhancements (v3.7.1)
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-SEM-*` / `REL-051` / `REL-054` / `US-SEM-*`. **Current P0:** `FR-SEM-07` / `REL-054`.
>
> **Evidence baseline (GREEN, no reopen) — small graph:** [`docs/semantic-search-mcp-verification-2026-07-17.md`](semantic-search-mcp-verification-2026-07-17.md). Live probe confirms HNSW+rerank, ontology, `kg_*` schema health, and graph-enriched context.
>
> **Mega-graph gap (P0, 2026-07-20):** [`docs/reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md`](reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md) — HNSW path OOMs when seed hydration still touches `all_elements()`. FR-SEM-01..05 remain agent UX / ops / release hygiene — **FR-SEM-07 is the mega safety fix**.
**Policy:**
- Keep shipped retrieval path (FR-HNSW-D): embed → HNSW top-k → optional rerank → optional graph traverse
- Do **not** treat MCP live smoke as a substitute for `cargo test --release --features embeddings` (`hnsw_recall_e2e`, `embeddings_state_e2e`, `vector_engine_e2e`, `ontology_e2e`, `mcp_tools_full_tests`, …)
- Prefer fixing default budgets + honesty over raising global caps that defeat the token mission
| ID | Priority | Requirement |
|----|----------|-------------|
| FR-SEM-01 | Should Have | Dual token accounting: top-level delivered `tokens` + `_token_budget.{max,actual,truncated}` always coherent; docs/skills teach 3–4× budget when `truncated: true` |
| FR-SEM-02 | Should Have | Explicit `max_tokens_for_tool` for `concept_search` and `kg_semantic_context` (≥ sibling `kg_*`, target 2k–4k); stop silent default-1000 truncation for those tools |
| FR-SEM-03 | Should Have | MCP HTTP resilience for long read-only semantic tools — document one-shot client retry; harden keep-alive / stale-listener ops path (see MCP HTTP stability analysis) |
| FR-SEM-04 | Should Have | Formal **live MCP semantic smoke** checklist (Docker `project=/workspace`) as release *complement*; template = verification docs 2026-07-17 / **2026-07-18** |
| FR-SEM-05 | Could Have | Optional file-diversity / MMR post-filter after HNSW+rerank so top-k is not ≥70% one file by default collapse |
| FR-SEM-06 | Must Have | Path filter in `FilterPolicy`: always drop `embed/assets/`; query-gate `src/benchmark/` unless query contains "benchmark" (Probes G/H) |
| FR-SEM-07 | Must Have (**P0**) | **Mega-safe HNSW semantic path:** `semantic_search` / `kg_semantic_context` (and any helper they call for seed hydration) must **not** invoke unbounded `all_elements()` or load the full element set into RAM on graphs above `LEANKG_MAX_CACHE_ELEMENTS`. Hydrate ANN hit QNs via keyed/paginated DB reads only. Peak RSS must fit documented MCP `mem_limit` (compose default 6g; effective cgroup may be lower) without OOM/restart. Small-graph HNSW path must not regress. |
| REL-051 | Should Have | Release note: live semantic smoke executed (or waived with reason) alongside embeddings cargo suite — **DONE** 2026-07-18 |
| REL-054 | Must Have (**P0**) | Live mega smoke gate: on `/workspace-other` (or equivalent), `semantic_search` + `kg_semantic_context` succeed without OOM/HTTP drop; record peak RSS + latency in a `docs/reports/` note. Complements REL-051 (small-graph) |
| FR-ONT-MEGA-01 | Must Have (**P0**) | **Mega-safe `concept_search`:** resolve `code_refs` via keyed/path-prefixed Cozo queries with `:limit`; name fallback via `search_by_name_typed`. Ban `load_indexed_code_elements` on the hot path. |
| FR-GF-MEGA-01 | Must Have (**P0**) | **Mega-safe `query_graph` / `shortest_path`:** keyed `resolve_to_qualified` (no `all_elements`); BFS via frontier-local relationship fetch (no `all_relationships`). |
| FR-CL-MEGA-01 | Must Have (**P1**) | **Mega `get_clusters` serve:** when live Louvain is refused, return clusters from precomputed `cluster_id`/`cluster_label`; else structured empty + offline-assign hint. |
| REL-055 | Must Have (**P0**) | Live mega smoke: `concept_search` + `query_graph` + `get_clusters` on `/workspace-other` without OOM/timeout; no `all_elements`/`all_relationships` WARN on those paths. |
**Won't Do (this track):**
- Replacing Cozo HNSW default before FR-VE-GATE callers honor LocalEngine
- Treating transient HTTP flakes as ANN / ontology product failures
- Dropping truncation entirely (mission is still lean tokens)
- Raising OrbStack/host VM RAM alone as the “fix” for `all_elements()` on mega HNSW (ops headroom helps; code must still stop full-graph dumps)
### 5.16 Day-2 Embed Resume / Resource Gate (v3.7.2) — **DONE (prior P0)**
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-EMBED-RESUME-*` / `US-EMBED-*` / `REL-052` / `FR-HNSW-E`.
>
> **Gap:** FR-HNSW-E delivered the incremental *filter*, but day-2 runs can still (a) drop+rebuild HNSW even when `to_embed` is empty, (b) lack mid-run checkpoint clarity, (c) look like a full cold restart after Docker remounts or when embed is turned on later. Product priority: **fix resource behavior before any new P1 feature work.**
**Policy (universal):**
- **If embed data exists → resume. If none → cold/fresh.** Default for every Docker/CLI embed entry path.
- Default `embed` mode remains **incremental**; `--full` / `LEANKG_EMBED_BACKGROUND_FULL=1` / `LEANKG_FORCE_REINDEX=1` are the only intentional full rebuilds / wipes
- Persist vectors + `embedding_state` in RocksDB (named volume) / SQLite `.leankg` — container recreate or “turn embed on” must not imply wipe
- Day-2 success = skip fresh + avoid needless HNSW rebuild + resume after interrupt
- Reclassify **FR-HNSW-E** as **PARTIAL** until FR-EMBED-RESUME-* close the gap
| ID | Priority | Requirement |
|----|----------|-------------|
| FR-EMBED-RESUME-01 | Must Have | Standalone Docker / CLI `embed --wait` loads existing `embedding_state` + vectors from RocksDB; second unchanged run skips `fresh` QNs (prove with logs: `skipped_fresh`, `embedded≈0`) |
| FR-EMBED-RESUME-02 | Must Have | When `to_embed` is empty and orphan set is empty, **skip** HNSW drop + recreate; exit quickly with clear “nothing to embed” status |
| FR-EMBED-RESUME-03 | Must Have | Mid-run durability: committed `fresh` rows survive kill/restart; next run resumes dirty-only (document lock + status files) |
| FR-EMBED-RESUME-04 | Must Have | Indexer must not force a full re-embed on no-op / identical content: mark stale only for QNs whose embeddable `content_hash` changed (audit full-index `mark_stale_for_qualified_names` blast radius) |
| FR-EMBED-RESUME-05 | Must Have | Day-2 SLA evidence: unchanged mega-graph second pass finishes with near-zero ONNX batches and wall time **≪** cold pass (document threshold in release note; target minutes not hours) |
| FR-EMBED-RESUME-06 | Must Have | **All Docker embed-on paths** (`LEANKG_EMBED_BACKGROUND`, `LEANKG_EMBED_ON_BOOT` / `embed_if_needed`, `LEANKG_DOCKER_SETUP`, `docker-up.sh`) share the same resume-vs-cold rule: existing data → incremental; no data → cold. Never wipe on enable. Document in `entrypoint.sh` / compose / AGENTS |
| FR-EMBED-RESUME-07 | Must Have | MCP/FG in-process path: cheap resume preflight (vector/state counts, no mega `all_elements` just to skip); zero-dirty → no ONNX/HNSW; small dirty → prefer incremental HNSW puts; status exposes `skipped_fresh` / `vectors_existing` |
| FR-EMBED-TOGGLE-01 | Must Have | MCP `embed_control` actions `on` / `off` / `status`; idle-gated arm; cooperative cancel; Admin for mutating actions |
| FR-EMBED-PARTIAL-01 | Must Have | Default MCP embed `mode=partial`: duty-cycle batches, yield on MCP activity, soft RSS ≤ `LEANKG_EMBED_RSS_FRACTION` (default 0.40) of container budget clamped by `LEANKG_EMBED_MAX_MB` |
| REL-052 | Must Have | Release gate: day-2 resume proven on named RocksDB volume for standalone `embed --wait` **and** at least one Docker MCP embed-on path (unit + e2e + optional live smoke) |
| FR-HNSW-E | Must Have (PARTIAL) | Keep incremental filter; remaining work tracked under FR-EMBED-RESUME-* |
**Won't Do (this track):**
- Deleting RocksDB / forcing `--full` as the “fix” for day-2 slowness
- Reopening Redis/FalkorDB as cold-write accelerators
- Blocking MCP boot on embed (FR-EMBED-R1 still stands)
- Treating “embed turned on in Docker” as a signal to wipe or full-rebuild
### 5.17 Mega-graph MCP auto-index + embed ops (v3.7.3)
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-MG-AUTO-*` / `FR-OPS-EMBED-*` / **P0** `FR-SEM-07` / `REL-054`.
>
> **Evidence:** [`docs/reports/embed-3-workspaces-2026-07-17.md`](reports/embed-3-workspaces-2026-07-17.md) — large multi-repo mounts trigger MCP incremental reindex on every start (RocksDB writes do not bump `.leankg/leankg.db` mtime), OOMs under 2g `mem_limit`, and flake semantic HTTP. `docker-compose.embed.yml` `cpus: "6"` failed on 5-vCPU hosts.
>
> **2026-07-20 follow-up:** Auto-index skip + compose CPU/mem knobs are **not sufficient** for mega HNSW query — see **FR-SEM-07** / **US-SEM-06** (P0). Evidence: [`docs/reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md`](reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md).
| ID | Priority | Requirement |
|----|----------|-------------|
| FR-MG-AUTO-01 | Must Have | Honor `LEANKG_SKIP_FRESHNESS_CHECK=1` in MCP `auto_index_if_needed` (no wipe). Document ops: `LEANKG_AUTO_INDEX=0`, `auto_index_on_start: false`, MCP `mem_limit: 6g` / `mem_reservation: 3g` / `cpus: "6"` for mega-graphs (~147k vectors). Follow-up: RocksDB manifest mtime vs `leankg.db` |
| FR-OPS-EMBED-CPU | Must Have | `docker-compose.embed.yml`: `cpus: "6"`, `mem_reservation: 3g`, `mem_limit: 10g`. MCP `docker-compose.rocksdb.yml`: multi-project default `cpus: "6"`, `mem_reservation: 3g`, `mem_limit: 6g` (override down for single-project Local 2g KPI) |
| FR-SEM-07 / REL-054 | Must Have (**P0**) | Cross-link: mega-safe HNSW query path — owned in §5.15; ops evidence still recorded under mega-graph reports |
**Won't Do:** `LEANKG_FORCE_REINDEX=1` as the fix for stale mega-graphs (wipes data).
### 5.18 MCP Tool Surface Rationalization (v3.7.4)
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-SURF-*` / `US-SURF-*` / `REL-053`.
>
> **Evidence:** Live MCP schemas + `src/mcp/handler.rs` / `tools.rs` review 2026-07-18. Registry baseline ≈85 tools. Fact corrections: `semantic_search` is dual-path (HNSW when embeddings exist); `search_code` is ontology-first on mega-graphs; `wake_up` = L0+L1 cached text (not L0 alone).
**Policy:**
- Prefer-order docstrings before deletes; deletes before merges
- Hard-delete only strict subsets / zero-value tools
- Soft-deprecate with an explicit replacement tool name in the schema
- Keep ops bootstrap tools (`mcp_status`, `mcp_init`, `mcp_index`, `mcp_index_docs`, `mcp_install`) — do not hide behind “deprecate all mcp_*”
- Update `tests/redundant_tools_matrix.rs` when the roster changes
- Recount surface size from `ToolRegistry::list_tools()` before quoting externally
| ID | Priority | Requirement |
|----|----------|-------------|
| FR-SURF-01 | Must Have | Fix `semantic_search` tool description: document dual path (HNSW+cross-encoder when embeddings index exists; else ontology-first `safe_discover`). Never claim ANN-only |
| FR-SURF-02 | Must Have | Add prefer-order one-liners to schemas for `concept_search`, `search_code`, `semantic_search`, `kg_semantic_context`, `kg_context` (match §3.16). Sync AGENTS / using-leankg skill |
| FR-SURF-03 | Must Have | Remove `mcp_hello`, `mcp_impact`, `get_doc_for_file` from `ToolRegistry` + handlers; update `tests/redundant_tools_matrix.rs` and any docs that reference them |
| FR-SURF-04 | Should Have | Soft-deprecate `wake_up`: description marks deprecated and points to `get_overview_context` (not `load_layer(L0)` alone) |
| FR-SURF-05 | Should Have | Soft-deprecate `search_by_environment`: description points to `env=` on primary search / `kg_*` tools |
| FR-SURF-06 | Could Have | Mega-safe `get_doc_structure` / `get_doc_tree` (no unbounded `all_elements()`); optional merge behind `format: "tree" \| "list"` after safety |
| REL-053 | Should Have | Release note: tool surface shrink after FR-SURF-03 (registry count before/after from `list_tools`) |
**Won't Do (this track):**
- Deprecating or renaming `mcp_status` / `mcp_init` / `mcp_index` / `mcp_index_docs` / `mcp_install` for cosmetic `get_*` consistency
- Replacing `wake_up` with `load_layer(layer="L0")` alone
- Merging the intentional search triple or semantic triple into one tool
- Quoting a “64 → 57” shrink without recounting the live registry
### 3.17 UI v2 — GitNexus Shell Adapted (US-UI2) — v3.7.9
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-UI2-*` / `FR-UI2-*` / `REL-056` / `REL-057` / `REL-060` / `REL-061`.
> **Design:** [`docs/erd/ui-v2-erd.md`](erd/ui-v2-erd.md).
> **Separate from:** Track E 3D `graph-ui/` (`REL-041` / `US-CBM-E1`).
| ID | Priority | Story |
|----|----------|-------|
| US-UI2-01 | Must Have | As a developer, I open `ui-v2` against `leankg serve` and explore the graph in Force, Tree, or Circles layout |
| US-UI2-02 | Must Have | As a developer, I filter node/edge types (defaults Service/Folder/File/Function) and browse a file tree of loaded nodes |
| US-UI2-03 | Must Have | As a developer, I select a **content-bearing** node (File/Function/Method/Class/…) and see syntax-highlighted source via `/api/file`; Service/Folder/Directory selection does **not** call `/api/file` |
| US-UI2-04 | Must Have | As a developer, I search via `/api/search` and run raw queries via QueryFAB `/api/query` |
| US-UI2-05 | Must Have | As a developer on a mega-graph, the UI skips full canvas load and offers “Load graph anyway” |
| US-UI2-06 | Must Have | As a developer, Query FAB default mode runs NL `query_graph`; Advanced mode keeps raw Cozo |
| US-UI2-07 | Must Have | As a company, ui-v2 is the default explorer embedded in `leankg serve` / Docker (cutover from Phase-1-only `ui-v2/`) |
| US-UI2-08 | Should Have | As a developer, I filter communities via a cluster legend (Graphify sidebar parity) |
| US-UI2-09 | Should Have | As an ops engineer, incidents / env / conflicts panels from legacy `ui/` are available in ui-v2 |
| US-UI2-10 | Must Have | As a developer on a multi-service topology, I double-click a Service/Folder/Directory and the canvas **replaces** with that path’s expand-service subgraph (`all=true`); breadcrumbs return me to overview |
| US-UI2-11 | Must Have | As a developer on a large expand (e.g. multi-repo workspace), I see the first **500** nodes and can **Load more (+200)** to **merge** additional pages into the same graph (500→700→…) without replacing |
| US-UI2-12 | Must Have | As a developer, the left Explore sidebar shows a **folder + file** tree (not files-only); `src` sorts above `examples`; double-click a folder drills the graph into that path |
**Phase 1 out of scope (closed):** browser LLM agent, analyze/upload, Processes Mermaid.
**Phase 2 (this revision):** NL Query FAB + cutover + cluster legend + ops panels (US-UI2-06..09).
### 3.18 Procedural Ontology Auto-Update (US-ONT-PROC) — v3.7.9 **P0**
> **Tasks:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-ONT-PROC-*` / `FR-ONT-PROC-*` / `REL-059`.
> **Gap evidence:** Procedural layer works (`kg_ontology_status`: 10 workflows / 48 steps; `kg_trace_workflow` live) but only refreshes via manual `leankg ontology sync` or Docker boot sync (`entrypoint.sh`). No in-process watch; boot skip marker ignores `workflows.yaml` mtime.
| ID | Priority | Story |
|----|----------|-------|
| US-ONT-PROC-01 | Must Have (**P0**) | As a developer using LeanKG MCP/serve, when I edit `ontology/workflows.yaml` (or concepts) or finish an index that changes step `code_refs`, **I want** procedural ontology to update **without** a manual sync or container restart, **so that** `kg_trace_workflow` stays accurate while I work |
**Acceptance (US-ONT-PROC-01):**
- **Given** `mcp-http` or `leankg serve` is running with ontology loaded, **When** I change a workflow step name or `code_refs` in `workflows.yaml` and wait the debounce window, **Then** `kg_trace_workflow` returns the new content without restarting the process.
- **Given** boot with an existing `.leankg/ontology_synced` marker newer than `concepts.yaml` but older than `workflows.yaml`, **When** the container starts, **Then** sync is **not** skipped solely because of `concepts.yaml`.
- **Given** a successful `leankg index` / MCP index that changes files referenced by workflow steps, **When** the index completes, **Then** procedural nodes are refreshed (or a documented MCP `ontology_sync` / `ontology_control` action is available and used by default hooks).
- **Won't Do in P0:** LLM auto-extraction of new workflows from arbitrary code (manual/agent-authored YAML remains the source of truth).
### 5.19 UI v2 Graph Explorer (v3.7.10)
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-UI2-*` / `REL-056` / `REL-057` / `REL-060` / `REL-061`.
> **Evidence:** [`docs/reports/ui-v2-gitnexus-parity-*.md`](reports/) (required before claiming GitNexus parity).
| ID | Priority | Requirement |
|----|----------|-------------|
| FR-UI2-01 | Must Have | New `ui-v2/` Vite+React+Tailwind+Sigma app; do not modify `ui/` or `src/embed/` in Phase 1 |
| FR-UI2-02 | Must Have | 3-pane exploring shell: FileTree+Filters / GraphCanvas / Code panel + Header + StatusBar |
| FR-UI2-03 | Must Have | Layout modes Force / Tree / Circles (GitNexus-shell behavior) |
| FR-UI2-04 | Must Have | Data plane uses LeanKG REST envelope `{success,data,error}`: topology, expand-service, children, search, file, query, index/status, project/switch, clusters |
| FR-UI2-05 | Must Have | Preserve US-MG-03/04 filter defaults (`DEFAULT_NODE_TYPE_ORDER`, `DEFAULT_VISIBLE_LABELS`) |
| FR-UI2-06 | Must Have | Mega-graph skip via `decideSkipGraph` (~50k nodes) + Load anyway |
| FR-UI2-07 | Must Have | Vitest units (adapter, load-decision, constants, client, url-restore) + Playwright Phase-1 e2e matrix |
| FR-UI2-12 | Must Have | Double-click Service/Folder/Directory → `expandService(path, all=true)` **replaces** `kg` (not merge); CodePanel `/api/file` only for content-bearing types; breadcrumb back to topology; `/api/file` returns clear directory error |
| FR-UI2-13 | Must Have | Expand-service `?limit=`/`?offset=` + correct `hasMore`; UI default page 500; **Load more (+200)** **merges** by node/edge id into current `kg`; pagination cursor advances by requested limit |
| FR-UI2-14 | Must Have | Explore sidebar hierarchical Folders & files (`buildExplorerTree`); include Directory/Folder; synthesize parents from paths; prefer `src` over demos; folder double-click → `drillIntoPath` |
| REL-056 | Must Have | Parity report with Pass/Fail vs GitNexus exploring shell (agent/analyze = N/A Phase 2) |
| FR-UI2-08 | Must Have | Query FAB dual-mode: NL → `query_graph` / orchestrate; Advanced → raw Cozo `POST /api/query` |
| FR-UI2-09 | Must Have | Build ui-v2 into `src/embed/` (or equivalent); `leankg serve` + Docker Option A serve ui-v2 by default |
| FR-UI2-10 | Should Have | Cluster legend + show/hide filters wired to `/api/graph/clusters` |
| FR-UI2-11 | Should Have | Port incidents / env / conflicts panels from legacy `ui/` into ui-v2 |
| REL-057 | Must Have | Cutover evidence: smoke + screenshots that embed/Docker serves ui-v2 as default |
| REL-060 | Must Have | Proof: Service select does not 400 `/api/file`; double-click replaces graph with expand-service subgraph |
| REL-061 | Must Have | Proof: expand page 500 then Load more grows canvas (merge); sidebar folders/files update |
**Won't Do (Phase 1 residual):** LangChain in-browser agent; GitNexus `/api/analyze` clone; Track E R3F 3D.
### 5.20 Company cost / competitive ROI (v3.7.8)
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `US-COST-*` / `FR-COST-*` / `REL-058`.
> **Narrative:** §1.1. **Evidence target:** [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md) + a short manager brief under `docs/reports/`.
| ID | Priority | Requirement |
|----|----------|-------------|
| US-COST-01 | Must Have | As an engineering manager, I can read a one-pager that shows why LeanKG reduces AI agent cost vs grep/cat and vs Graphify at company monorepo scale |
| FR-COST-01 | Must Have | Publish ROI brief: token/tool-call floors (Section 9), multi-repo Docker TCO, mega-graph safety, ops/traceability differentiators; link §1.1 queue |
| FR-GF-21 | Must Have | CLI/MCP `export html` — single-file bounded subgraph/community; document node budget |
| FR-GF-22 | Must Have | README / AGENTS / using-leankg skill lead with path · explain · query; demote full tool wall |
| FR-GF-23 | Should Have | Expand `leankg install` platforms (start Cursor + Claude + Codex; grow toward Graphify breadth) |
| FR-GF-24 | Must Have | Always-on graph-first rules/hooks: nudge before grep/Read; optional strict first-Read redirect; document for Cursor + Claude Code |
| REL-058 | Must Have | Manager ROI brief checked into `docs/reports/` and linked from README competitive section |
**Won't Do:** Claiming LOCOMO memory-suite wins; multimodal ingest as a cost strategy.
### 5.21 Procedural ontology auto-update (v3.7.9) — **CURRENT P0**
> **FR checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter `FR-ONT-PROC-*` / `US-ONT-PROC-*` / `REL-059`.
> **Related:** `FR-A02` (docs/automation) stays P1; this section is the **runtime** auto-update.
> **Baseline:** Boot sync in `entrypoint.sh` + CLI `leankg ontology sync` only; no YAML watch during MCP use.
| ID | Priority | Requirement |
|----|----------|-------------|
| FR-ONT-PROC-01 | Must Have (**P0**) | While `mcp-http` / `leankg serve` runs, watch `ontology/workflows.yaml` and `ontology/concepts.yaml` (project + configured source dir); debounce (≥1s) and run idempotent ontology sync into the served DB without dropping the HTTP listener |
| FR-ONT-PROC-02 | Must Have (**P0**) | Docker/boot skip marker must consider **both** `concepts.yaml` and `workflows.yaml` mtimes (and force re-sync when either is newer than `.leankg/ontology_synced`) |
| FR-ONT-PROC-03 | Must Have (**P0**) | After successful index (CLI or MCP), refresh procedural ontology (re-bind step `code_refs` / re-sync YAML) or expose MCP `ontology_control(action=sync\|status)` and invoke it from the index completion path |
| REL-059 | Must Have (**P0**) | Live smoke documented in `docs/reports/`: (1) edit workflow step → `kg_trace_workflow` updates without restart; (2) boot with stale workflows.yaml triggers sync; (3) sync never blocks `/health` beyond existing ontology timeout policy |
**Won't Do (P0):** Automatic LLM generation of new workflows from code; replacing YAML as source of truth; blocking MCP bind on sync (keep timeout/skip escape).
---
## 6. Technical Architecture / HLD
### 6.1 Technology Stack
| Component | Technology | Version |
|-----------|------------|---------|
| Core Language | Rust | 1.70+ (edition 2021) |
| Database | CozoDB (SQLite / RocksDB) + native `::hnsw` | 0.7.6 |
| Code Parsing | tree-sitter | 0.25 |
| MCP Server | rmcp (Rust MCP library) | 1.2 |
| CLI Framework | Clap | 4 |
| Web UI | Axum | 0.7 |
| Async Runtime | Tokio | 1 |
| File Watching | notify | 7 |
| Parallel Processing | rayon | 1.10 |
| Markdown Parsing | pulldown-cmark | 0.12 |
| Auth (API keys) | Argon2 | 0.5 |
| CORS | tower-http | 0.6 |
### 6.2 Data Model
```
CodeElement:
- qualified_name: string (PK) - format: "path/to/file.rs::function_name" or "path/to/dir/" for directories
- element_type: string - directory | file | function | class | import | export | pipeline | pipeline_stage | pipeline_step | terraform | cicd | document | doc_section
- name: string
- file_path: string
- line_start: int
- line_end: int
- language: string
- parent_qualified: string? (nullable)
- cluster_id: string? (nullable)
- cluster_label: string? (nullable)
- metadata: JSON (includes signature, headings, ci_platform, child_count for directories, etc.)
Relationship:
- source_qualified: string (FK)
- target_qualified: string (FK)
- rel_type: string - imports | calls | references | documented_by | tested_by | tests | contains | defines | implements | implementations | tunnel | decided_about
- confidence: float (0.0-1.0)
- metadata: JSON
Indexes: rel_type_index, target_qualified_index
> **Folder-as-Graph Design (MemPalace-inspired):** Directories are first-class `directory` nodes in the graph. The `contains` edge is overloaded to represent the full hierarchy: `directory → directory`, `directory → file`, `file → function/class`. This mirrors MemPalace's wing → room → closet → drawer spatial architecture:
>
> | MemPalace | LeanKG | Edge |
> |-----------|--------|------|
> | Wing (project/person) | Top-level directory (`src/`, `docs/`) | `contains` |
> | Room (topic) | Sub-directory (`src/graph/`, `src/mcp/`) | `contains` |
> | Closet (summary) | File (`src/graph/query.rs`) | `contains` |
> | Drawer (verbatim) | Function/class within file | `contains` |
>
> Benefits:
> - **Impact analysis at directory level:** "What modules are affected if I change anything in `src/indexer/`?"
> - **Cluster-to-directory alignment:** Auto-detect when a Leiden cluster maps to a physical directory
> - **Wake-up context includes module map:** L0/L1 can list top-level directories as the "palace wings"
> - **Tunnel edges between directories:** Link `src/auth/` and `src/middleware/` when they share domain concepts
> - **Folder search:** `query_file` and `search_code` can scope to directory nodes
BusinessLogic:
- element_qualified: string (PK, FK)
- description: string
- user_story_id: string? (nullable)
- feature_id: string? (nullable)
ContextMetric:
- tool_name: string (indexed)
- timestamp: int (indexed)
- project_path: string (indexed)
- input_tokens: int
- output_tokens: int
- output_elements: int
- execution_time_ms: int
- baseline_tokens: int
- baseline_lines_scanned: int
- tokens_saved: int
- savings_percent: float
- (+ optional fields: correct_elements, total_expected, f1_score, query_pattern, query_file, query_depth, success, is_deleted)
QueryCache:
- cache_key: string (unique)
- value_json: string
- created_at: int
- ttl_seconds: int
- tool_name: string
- project_path: string
- metadata: JSON
ApiKey:
- id: string (UUID)
- name: string
- key_hash: string (Argon2)
- created_at: int
- last_used_at: int?
- revoked_at: int?
```
### 6.3 Module Map
```
src/
├── main.rs # CLI entry point (30+ commands; includes lsp_resolve, check_consistency, tunnels, prs, clones, reflect)
├── lib.rs # Library exports (registers modules below)
├── cli/ # Clap command enum + ShellRunner
├── config/ # ProjectConfig, IndexerConfig (typed_resolve flag), DocConfig, McpConfig
├── db/ # CozoDB models, schema, operations, API key store, valid_from/valid_to fields
├── doc/ # DocGenerator, template rendering, wiki generation
├── doc_indexer/ # Documentation indexing (docs/ → documented_by edges)
├── graph/ # GraphEngine, queries, context, traversal, clustering, cache (incl. hot-path cache), export (HTML/SVG/GraphML/Neo4j/snapshot), clones, tunnels
├── indexer/ # tree-sitter parsers (17), extractors (incl. dart/swift/xml/vue/svelte/sql_ddl/rationale/routes), git analysis, Terraform, CI/CD
├── lsp/ # NEW — generic LSP bridge (bridge.rs, client.rs, config.rs, mod.rs); per-(language, workspace) client cache
├── mcp/ # MCP tools (85), handler (resolve_with_lsp + agent_focus + diary + …), server (rmcp), auth, write tracker
├── orchestrator/ # Query orchestration with intent parsing and persistent cache
├── compress/ # RTK-style compression: 8 read modes, response/shell/cargo/git compressors, entropy analysis
├── web/ # Axum web UI (20+ routes, embedded HTML/CSS/JS)
├── api/ # REST API handlers, auth middleware
├── watcher/ # notify-based file watcher for auto-indexing
├── hooks/ # Git hooks (pre-commit, post-commit, post-checkout, GitWatcher)
├── benchmark/ # Benchmark runner (LeanKG vs OpenCode/Gemini/Kilo)
├── ontology/ # Concept + procedural ontology (concepts.yaml, workflows.yaml) — kg_* tools
├── embeddings/ # Semantic embeddings → CozoDB `embedding_vectors` + `::hnsw` (feature-gated; product focus)
├── retrieval/ # embed → HNSW ANN → rerank → graph traverse
├── embed.rs # Legacy/compat embedding wrappers (prefer `embeddings/`)
├── budget.rs # Per-tool token / RSS / wall-clock budget enforcement
├── gc.rs # MemoryGuard for long-running MCP daemons
├── obsidian/ # Obsidian-vault doc adapter
├── registry.rs # Global repository registry (multi-repo management)
└── runtime.rs # Tokio runtime utilities
```
### 6.4 HLD — System Overview (merged from `hld-leankg.md`)
```
+-----------------------------------------------------+
| LeanKG Backend |
| (Axum + CozoDB / RocksDB) |
| |
| +--------------+ +--------------+ +----------+ |
| | production | | staging | | local | |
| | namespace | | namespace | |namespace | |
| +--------------+ +--------------+ +----------+ |
| |
| +-----------------------------------------------+ |
| | CozoDB (Datalog) + HNSW embeddings (semantic ANN focus) | |
| +-----------------------------------------------+ |
+-----------------------------------------------------+
^ ^ ^
| | |
+------+------+ +--------+-----+ +------+------+
| MCP server | | REST API | | Web UI |
| stdio/HTTP | | /api/... | | 2D (+3D E) |
+------+------+ +--------+-----+ +-------------+
| |
+------+------+ +--------+---------------------+
| AI assistants| | CI/CD hooks / GitHub Actions |
+--------------+ +------------------------------+
```
### 6.5 HLD — Component Design
**Data layer:** `env` on `code_elements` / `relationships`; incidents + service metadata tables; all queries filter by env (default `local`).
**Graph engine tools (v2):** `get_service_context`, `find_env_conflicts`, `query_incidents`, env-aware impact.
**MCP auth headers (HTTP):** `X-LeanKG-Token` / Bearer; optional engineer + env headers.
**CLI (v2):** `leankg incident add|list|show`, `leankg update`, note/pattern/env helpers as implemented.
**Vacuum scheduler:** tokio task; `LEANKG_VACUUM_INTERVAL_HOURS` (default 1, `0` disables); Sqlite VACUUM; RocksDB debug no-op; invalidate caches after success.
**Ontology self-test:** `kg_self_test` + HTTP startup WARN (non-gating) for arity/schema drift on `kg_*` tools.
### 6.6 HLD — Key Data Flows
**Incident contribution:** CLI/API → validate Incident → CozoDB → available to MCP queries.
**Env conflicts:** fetch service across envs → compare schema/config/endpoints/deploy → risk HIGH/MEDIUM/LOW.
**Vacuum:** boot → spawn loop → vacuum → log → sleep.
**kg_self_test:** bind HTTP → run OntologyQueryEngine::self_test → info if OK, warn per failure → still serve.
### 6.7 HLD — Implementation Phases (v2)
| Phase | Scope | Status |
|-------|-------|--------|
| 1 Data model & schema (`env`, Incident) | Schema | DONE |
| 2 Graph engine env/incident queries | Engine | DONE |
| 3 MCP tools + token budgets | MCP | DONE |
| 4 CLI incident/update | CLI | DONE |
| 5 Integration tests / CI template | Test/Docs | PARTIAL |
### 6.8 HLD — Interface Sketches
`query_incidents` input: `{service, pattern?, env, limit}` → incidents[].
`find_env_conflicts` input: `{service}` → conflicts[{type, detail, risk}].
`leankg incident add --title … --severity P1 --affected svc --env production`.
### 6.9 HLD — Risks & Mitigations
| Risk | Impact | Mitigation |
|------|--------|------------|
| Schema migration breaks v1 data | High | Default `env=local` for existing rows |
| Token budgets too tight | Medium | Configurable budgets + TOON |
| Scale to large multi-service graphs | High | RocksDB, caches, pagination |
| Concurrent writes | Medium | Cozo transactions |
| Unbounded DB growth | Medium | Hourly vacuum |
| Ontology arity drift → MCP -32603 | High | `kg_self_test` startup WARN |
| LocalEngine dual-write crash leaves dangling offsets | High | Append → fsync → Rocks commit → RAM; recovery skips incomplete (FR-VE-FS-*) |
| SIMD path SIGILL on older CPUs | High | Runtime feature detect + scalar fallback (FR-VE-RT-SIMD) |
| 2GB cgroup OOM during ANN warm | High | Auto-tune block cache + SQ8-only hot path (FR-VE-RT-MEM / BENCH-OOM) |
| Premature Cozo→LocalEngine default switch | Medium | Hard FR-VE-GATE before changing FR-HNSW-B default |
### 6.10 HLD — Optimized Local-First Vector Graph Engine (v3.7.0)
```
+---------------------------------------------+
| Retrieval API (unchanged MCP/CLI) |
| semantic_search / kg_semantic_context / … |
+----------------------+----------------------+
|
+----------------------v----------------------+
| Storage Factory (env / .env / leankg.yaml)|
| LocalEngine | CloudEngine (static enum) |
+----------+------------------+---------------+
| |
Local (ARM64/x86) | | Cloud (x86_64)
v v
+---------------------+----+ +--------+------------------+
| Tier 1 RocksDB | | Tier 1 TiKV |
| metadata + HNSW adj | | metadata + HNSW adj |
| mmap OFF, Zstd, pin L0 | | distributed KV |
+------------+-------------+ +--------+------------------+
| |
+------------v-------------+ +--------v------------------+
| Tier 2 SQ8/INT8 in RAM | | Tier 2 SQ8/INT8 in RAM |
| SIMD: NEON/AVX2/AVX-512 | | SIMD + full-core rayon |
| (leave 2 cores Local) | | (use 50-80% RAM) |
+------------+-------------+ +--------+------------------+
| |
+------------v-------------+ +--------v------------------+
| Tier 3 Flat binary | | Tier 3 Flat / object store|
| FP32 + source payload | | FP32 + source payload |
| post-filter read once | | post-filter read once |
+--------------------------+ +---------------------------+
Dual-write: Flat append → fsync → Tier1 offsets → Tier2 RAM update
GC: shadow page + delta sync when fragmentation > 30% (readers unblocked)
```
**Dynamic adaptation:** cgroups/`sysinfo` → RocksDB block cache; runtime CPU feature detect → SIMD lane; Local leaves 2 cores free.
**Migration:** Cozo `embedding_vectors:vec_idx` remains default until FR-VE-GATE; optional dual-run / shadow compare for recall before cutover.
---
## 7. MCP Tools (85 total — audited 2026-07-14 against `src/mcp/tools.rs` v0.17.9)
### Project Management (5)
| Tool | Description |
|------|-------------|
| `mcp_init` | Initialize LeanKG project |
| `mcp_index` | Index codebase |
| `mcp_index_docs` | Index docs directory |
| `mcp_install` | Create .mcp.json |
| `mcp_status` | Show index status |
### Impact & Dependency (6)
| Tool | Description |
|------|-------------|
| `mcp_impact` | Calculate blast radius |
| `get_impact_radius` | Affected files within N hops with confidence/severity |
| `detect_changes` | Pre-commit risk analysis |
| `get_dependencies` | Direct imports of a file |
| `get_dependents` | Files depending on target |
| `get_review_context` | Focused subgraph + review prompt |
### Code Search (7)
| Tool | Description |
|------|-------------|
| `search_code` | Search by name/type |
| `find_function` | Locate function definition |
| `query_file` | Find file by pattern |
| `get_callers` | Find callers of a function |
| `get_call_graph` | Bounded call chain |
| `get_code_tree` | Codebase structure |
| `find_large_functions` | Oversized functions by line count |
### Context & Compression (3)
| Tool | Description |
|------|-------------|
| `get_context` | AI-optimized file context |
| `ctx_read` | Read file with 8 compression modes |
| `orchestrate` | Smart query routing with cache |
### Testing & Docs (7)
| Tool | Description |
|------|-------------|
| `get_tested_by` | Test coverage info |
| `get_doc_for_file` | Docs referencing code element |
| `get_files_for_doc` | Code elements in a doc |
| `get_doc_structure` | Documentation directory structure |
| `get_doc_tree` | Doc tree with hierarchy |
| `generate_doc` | Generate documentation |
| `find_related_docs` | Docs related to code change |
### Traceability (2)
| Tool | Description |
|------|-------------|
| `get_traceability` | Full traceability chain |
| `search_by_requirement` | Code for a requirement |
### Clustering & Graph (3)
| Tool | Description |
|------|-------------|
| `get_clusters` | Functional communities |
| `get_cluster_context` | Cluster symbols and dependencies |
| `generate_graph_report` | Comprehensive graph analysis |
### Export & Utility (2)
| Tool | Description |
|------|-------------|
| `export_graph` | Export in json/html/svg/graphml/neo4j |
| `mcp_hello` | Health check / debug |
### 7.5 TOON Response Templates
All MCP tool responses use TOON (Token-Oriented Object Notation) format by default for ~40% token reduction. See [TOON Specification](https://github.com/toon-format/toon) for details.
**Response Format Envelope:**
```
{
status: ok|error
tool: <tool_name>
format: toon|json
tokens: <token_count>
data: <response_data>
}
```
**TOON Format Examples:**
1. **Search/Query Results:**
```
{
status: ok
tool: search_code
format: toon
tokens: 156
data:
results[3]{qualified_name,type,language}:
src/main.rs::main,function,rust
src/lib.rs::init,function,rust
src/cli.rs::run,function,rust
}
```
2. **Impact Radius:**
```
{
status: ok
tool: get_impact_radius
format: toon
tokens: 203
data:
impact[4]{qualified_name,type,severity,confidence}:
src/main.rs::main,function,WILL_BREAK,1.0
src/lib.rs::init,function,LIKELY_AFFECTED,0.85
src/config.rs::load,function,LIKELY_AFFECTED,0.72
tests/main_test.rs::test_main,test,MAY_BE_AFFECTED,0.31
}
```
3. **Dependencies/Dependents:**
```
{
status: ok
tool: get_dependencies
format: toon
tokens: 98
data:
dependencies[2]{qualified_name,type}:
src/lib.rs,file
src/config.rs,file
}
```
4. **Call Graph:**
```
{
status: ok
tool: get_call_graph
format: toon
tokens: 187
data:
calls[3]{from,to,depth}:
src/main.rs::main,src/lib.rs::init,1
src/main.rs::main,src/cli.rs::run,1
src/lib.rs::init,src/config.rs::load,2
}
```
5. **Context/Compression:**
```
{
status: ok
tool: get_context
format: toon
tokens: 412
data:
context:
sig[1]: src/main.rs::main->()
imports[2]: src/lib.rs,src/config.rs
calls[1]: src/lib.rs::init
}
```
6. **Cluster/Graph Data:**
```
{
status: ok
tool: get_clusters
format: toon
tokens: 234
data:
clusters[2]{id,name,members}:
c1,mcp_tools,12
c2,graph_core,8
}
```
**JSON Fallback:** Clients can request JSON format by adding `format=json` parameter to any MCP tool call.
---
## 8. Release Criteria
> **Release checklist + status:** [`prd-task-tracker.md`](prd-task-tracker.md) — filter Kind=`Release` or section 8.* / FR-VE-GATE / REL-052 / **REL-054**.
### 8.5 v3.7.2 Embed Resume Gate — **DONE (prior P0)**
> **Status:** **DONE** on PR [#81](https://github.com/FreePeak/LeanKG/pull/81) / [#86](https://github.com/FreePeak/LeanKG/pull/86) — tracker `REL-052` + `FR-EMBED-RESUME-*` / `US-EMBED-*` / `FR-EMBED-TOGGLE-01`.
>
> **Current P0** is mega-graph HNSW query safety — see **§8.6** (`REL-054` / `FR-SEM-07`).
**Gate checklist:**
1. Unchanged second `embed --wait` on RocksDB volume: `embedded≈0`, `skipped_fresh` dominates (FR-EMBED-RESUME-01 / US-EMBED-01) — **DONE**
2. Zero-dirty path skips HNSW drop/rebuild (FR-EMBED-RESUME-02 / US-EMBED-03) — **DONE**
3. Kill mid-run → resume dirty-only (FR-EMBED-RESUME-03 / US-EMBED-02) — **DONE**
4. No-op index does not stale-all (FR-EMBED-RESUME-04) — **DONE**
5. Day-2 wall time ≪ cold on mega-graph; release note records numbers (FR-EMBED-RESUME-05 / REL-052) — **DONE** (MCP idle resume evidence 2026-07-20: `skipped_fresh: 147175`, ~2.7 s)
6. Docker embed-on resumes existing data; cold only when empty (FR-EMBED-RESUME-06 / US-EMBED-04) — **DONE**
### 8.6 v3.7.5 Mega-graph HNSW Semantic Safety Gate (**CURRENT P0**)
> **Status:** **OPEN** — tracker `US-SEM-06` / `FR-SEM-07` / `REL-054`.
>
> Ship mega-safe HNSW `semantic_search` / `kg_semantic_context` before treating mega Docker MCP as production-ready for NL retrieval. Small-graph probe (REL-051) does **not** close this gate.
>
> **Failure baseline:** [`docs/reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md`](reports/main-a89a2cc-docker-mega-tool-test-2026-07-20.md) — mega OOM; `/workspace` HNSW GREEN.
**Gate checklist:**
1. No unbounded `all_elements()` (or equivalent full-graph RAM load) on HNSW semantic seed hydration (FR-SEM-07) — **NOT_DONE**
2. `semantic_search` on `/workspace-other` returns without OOM/HTTP drop; `/health` stays ok (US-SEM-06 / REL-054) — **NOT_DONE**
3. `kg_semantic_context` same mega stability AC — **NOT_DONE**
4. Small `/workspace` HNSW+rerank remains GREEN (no regression) — **PASS** (baseline 2026-07-20)
5. Report peak RSS + latency in `docs/reports/` after fix (REL-054) — **NOT_DONE**
## 9. Non-Functional Requirements
| Metric | Target | Status |
|--------|--------|--------|
| Cold start time | < 2 seconds | TBD |
| Indexing speed | > 10,000 lines/second (parallel via rayon) | TBD |
| Time-to-context (chunks + deps JSON) P95 | **< 100ms** | DONE (US-VE-02 — measured **0.086ms** P95) |
| ANN query P95 (1M SQ8, Local) | **< 50ms** | DONE (FR-VE-BENCH-Q — measured **0.055ms** Neon @ 1M) |
| Query response time (legacy general) | < 100ms | TBD |
| Memory usage (idle MCP) | **< 150MB** (was 100MB aspirational) | DONE (US-VE-01 — lean absolute ≈**65MB**; warm delta ≈**58MB**; CI gates `delta_ok`) |
| Memory usage (indexing) | < 500MB typical; Cloud may use 50–80% RAM for SQ8 | TBD |
| Survival under cgroup | **2GB hard** — never OOM-killed | DONE (FR-VE-BENCH-OOM — est. heap ≈1.06GB; live alloc under 2GB) |
| Disk I/O vs legacy mmap | ≥ **80%** fewer page faults / disk reads | DONE (FR-VE-BENCH-IO — **99.999%** modeled) |
| HNSW recall @ efSearch=50 vs FP32 BF | **> 90%** | DONE (FR-VE-BENCH-RECALL — measured **1.000** @ ef=50) |
| Agent token savings vs grep/cat | ≥ **60%** (stretch 61%) | DONE (FR-VE-BENCH-AB — measured **65.0%** @ 100 tasks) |
| Agent tool-call reduction vs baseline | ≥ **80%** (stretch 84%) | DONE (FR-VE-BENCH-AB — measured **84.6%**) |
| Agent time-to-resolution | ≥ **2×** faster | DONE (FR-VE-BENCH-AB — measured **2.50×**) |
| Agent task success rate | ≥ baseline | DONE (FR-VE-BENCH-AB) |
| detect_changes response time | < 2 seconds | TBD |
| get_context enhanced response size | < 4000 tokens | TBD |
| Batch insert size | 5000 rows/batch | DONE |
| Supported parser / extractor count | Tree-sitter + specialized extractors; **indexed walk ≈ 8 code langs + Android/XML/TF/CI** (Swift/Vue/Svelte/SQL modules unwired) | PARTIAL |
| MCP tool count | 85 tools (`src/mcp/tools.rs`) | DONE (audited 2026-07-14; still 85 on v0.19.0) |
| Cross-platform | Apple Silicon (ARM64) Local + Linux x86_64 Cloud | PARTIAL (FR-VE-ABS DONE; CloudEngine TiKV Tier-1 still stub root) |
| Token honesty (delivered vs actual) | When `truncated: true`, agents can read both figures; docs teach ≥3× budget | PENDING (FR-SEM-01) |
| Ontology-tool default budgets | `concept_search` / `kg_semantic_context` ≥ 2k (align with sibling `kg_*`) | PENDING (FR-SEM-02) |
| MCP HTTP semantic flake | Read-only semantic tools survive one transient socket drop via retry / hygiene | PENDING (FR-SEM-03) |
| Live semantic MCP smoke | Checklist run (or waived) as release complement to embeddings cargo tests | PENDING (FR-SEM-04 / REL-051) |
| Day-2 embed (unchanged graph) | Near-zero ONNX; wall time ≪ cold; minutes not hours on mega-graph | DONE local smoke + e2e (FR-EMBED-RESUME-01 / 02); mega-graph wall-time PARTIAL (FR-05) |
| Zero-dirty embed | No HNSW drop/rebuild when nothing to write | DONE (FR-EMBED-RESUME-02) |
| Embed interrupt resume | Already-`fresh` rows never re-inferred after kill/restart | DONE (FR-EMBED-RESUME-03) |
| Docker embed-on resume | Existing RocksDB embed data → resume; empty → cold only | PARTIAL — shared build path + AGENTS (FR-EMBED-RESUME-06 / US-EMBED-04); live Docker smoke optional |
---
## 10. Out of Scope
1. **Full multi-modal PDF/image/video graph ingest (Graphify-style)** - Code + docs + infra first; not a company cost lever
2. **Cloud SaaS hosting of LeanKG** - Self-hosted only (team HTTP MCP / RocksDB / **self-hosted TiKV CloudEngine** is in scope; managed multi-tenant SaaS is not)
3. **Multi-user collaborative editing of the graph** - Single writer per project DB; shared read via HTTP MCP is OK
4. **Plugin system** - Future consideration
5. **Raw Datalog query passthrough** - Security risk (except controlled `run_raw_query`)
6. **Replacing CozoDB/RocksDB with NetworkX-only primary store** - Snapshot/HTML export is additive
7. **Full 158-language / Pure-C rewrite (CBM chase)** - Selective languages only
8. **Split PRD/HLD documents** - This file is the only SoT for narrative/HLD; do not recreate `docs/requirement/prd-*.md` or `docs/design/hld-leankg.md`. Task lists/status live only in [`prd-task-tracker.md`](prd-task-tracker.md)
9. **Status tables / FR checkboxes inside this PRD** - Forbidden; use the tracker
10. **Redis/FalkorDB as cold-embed write accelerator** - Rejected v3.6.3; not revived by v3.7 vector engine
11. **Default cutover from Cozo HNSW before FR-VE-GATE** - Explicitly forbidden
12. **Replace Sigma live UI with vis.js-only static HTML** - Steal HTML *export*; keep live explorer
13. **36–40 language grammar race to match Graphify** - Wire tested extractors only; prefer typed resolve depth
14. **LLM auto-extraction of procedural workflows from arbitrary code (P0)** - YAML remains SoT for v3.7.9; auto-update means watch/sync/index-hook of authored ontology, not Graphiti-style LLM invent
---
## 11. Glossary
| Term | Definition |
|------|------------|
| Knowledge Graph | Graph structure storing entities and relationships from codebase |
| Code Indexing | Process of parsing code and extracting structural information |
| MCP Server | Model Context Protocol server for AI tool integration (rmcp) |
| Context Window | AI model's input capacity; LeanKG minimizes tokens needed |
| Business Logic Mapping | Linking code to business requirements |
| Qualified Name | Natural node identifier: `file_path::parent::name` format |
| Blast Radius / Impact Radius | All files affected by a change within N hops |
| Confidence Score | Float 0.0-1.0 indicating edge reliability |
| Confidence Label | EXTRACTED / INFERRED / AMBIGUOUS provenance |
| Severity Classification | WILL BREAK / LIKELY AFFECTED / MAY BE AFFECTED |
| Cluster | Functional community (Leiden) |
| God Node | High-degree hub concept |
| Environment Namespace | `local` / `staging` / `production` / `upcoming` partition of graph data |
| Incident Node | Structured outage/knowledge record linked to services |
| Vacuum Scheduler | Periodic SQLite VACUUM on long-lived MCP servers |
| RTK | Rust Token Killer — compression reducing LLM tokens |
| Orchestrator | Intent parsing + persistent cache |
| Global Registry | Multi-repo management for cross-project queries |
| Temporal Graph | Relationships with valid_from/valid_to |
| Wake-up Protocol | Minimal L0+L1 context at session start |
| HLD | High-Level Design — architecture and flows in Section 6.4–6.10 |
| LocalEngine | 3-tier local vector/graph backend (RocksDB + SQ8 RAM + flat payload) |
| CloudEngine | Same API as LocalEngine backed by TiKV (and cloud-scale RAM) |
| SQ8 / INT8 quantization | Down-casted vectors kept fully in RAM for SIMD ANN |
| Flat Payload File | Tier-3 append-only binary storing FP32 + source for post-filter |
| Dual-Write | Append → fsync → commit offsets → update RAM (crash-safe order) |
| FR-VE-GATE | Quality gate required before replacing Cozo HNSW as Local default |
| Token honesty | Delivered `tokens` vs `_token_budget.actual` when payloads are truncated |
| Live MCP semantic smoke | Docker HTTP probe of `semantic_search` / `concept_search` / `kg_*` — complements, does not replace, cargo embeddings tests |
---
## 12. References
- CozoDB: https://github.com/cozodb/cozo
- tree-sitter: https://tree-sitter.github.io/tree-sitter/
- MCP Protocol: https://modelcontextprotocol.io/
- rmcp: https://crates.io/crates/rmcp
- Leiden Algorithm: https://en.wikipedia.org/wiki/Leiden_algorithm
- MemPalace: https://github.com/milla-jovovich/mempalace
- Graphify: https://github.com/Graphify-Labs/graphify — [`docs/analysis/graphify-vs-leankg-2026-07-20.md`](analysis/graphify-vs-leankg-2026-07-20.md) (primary) · [`graphify-comparison-2026-07-13.md`](analysis/graphify-comparison-2026-07-13.md) (historical)
- codebase-memory-mcp: https://github.com/DeusData/codebase-memory-mcp — see Section 3.11 / 5.10
- Context enhancement analysis: `docs/analysis/enhancement-analysis-2026-07-09.md`
- Roadmap: `docs/roadmap.md`
- MCP tool reference: `docs/mcp-tools.md`
- CLI reference: `docs/cli-reference.md`
- Embed store how-it-works: `generated_docs/embed_store_how_it_works_2026-07-16.md`
- Semantic MCP live verification (2026-07-17): [`docs/semantic-search-mcp-verification-2026-07-17.md`](semantic-search-mcp-verification-2026-07-17.md)
- MCP HTTP stability analysis: `docs/analysis/mcp-http-stability-analysis-2026-05-05.md`
- **Task tracker (all US/FR/Release + status):** [`docs/prd-task-tracker.md`](prd-task-tracker.md) / [`prd-task-tracker.json`](prd-task-tracker.json)
---
*Last updated: 2026-07-17 (v3.7.1 — semantic MCP verification → US-SEM / FR-SEM enhancement backlog; crate 0.19.0)*