knot-server 0.7.0

# knot-server

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-2024-brightgreen.svg)](https://www.rust-lang.org)

**knot-server** is a distributed REST API and background task scheduler for managing and indexing Git repositories across a cluster. It sits on top of the core [knot](https://github.com/raultov/knot) indexing engine, transforming it from a single-machine CLI tool into a highly available, cluster-aware enterprise service.

See [CHANGELOG.md](CHANGELOG.md) for the full version history.

**knot-server** and the [knot](https://github.com/raultov/knot) indexing engine are in **beta**. Expect rough edges, occasional breaking changes, and indexing quirks as we approach a stable 1.0 release.

> ⚠️ **Upgrading to v0.2.9 (knot 1.5.1) — automatic full re-index.** knot 1.5.1
> bumps the on-disk index-state format (v3 → v4) and switches stored `file_path`
> values to repo-relative form. The older state is incompatible, so the **first
> sync of each repository after upgrading is a full re-index** (expect it to take
> as long as the initial index). No manual action is required — knot-server
> detects the stale state, discards it, and rebuilds the Neo4j/Qdrant entries
> automatically. See the [CHANGELOG](CHANGELOG.md) for details.
> 
> ⚠️ **Upgrading to v0.2.19 (knot 1.5.6) — re-index Groovy repos.** knot 1.5.6
> adds Groovy property accessor synthesis, bare property declarations, and
> parser/Javadoc fixes. **Existing Groovy repositories must be re-indexed**
> (`POST /api/repos/{id}/sync`) for the new entities and `OVERRIDES` edges
> to materialize. See the [CHANGELOG](CHANGELOG.md) for details.

With `knot-server`, you can register Git repositories via a REST API, trigger automatic codebase indexing through webhooks (GitHub, GitLab, Bitbucket), and query the vector (Qdrant) and graph (Neo4j) databases—all while coordinating work safely across multiple server instances via NFS/EFS workspace locks.

<p align="center">
  <img src="docs/demo-graph.webp" alt="knot-server 3D codebase graph visualizer" width="49%">
  <img src="docs/demo-swagger.webp" alt="knot-server Interactive Swagger UI" width="49%">
</p>

---

## 🧮 Token Efficiency — Measured, Not Claimed

An LLM agent exploring an unfamiliar codebase pays for every byte it reads. Without an index it greps and then reads whole files; with `knot-server`'s REST APIs, it receives targeted answers, bringing the exact same token-saving performance of the core `knot` engine to multi-node enterprise environments. The difference was measured on **three real indexed repositories** across nine realistic exploration tasks:

| Repo | Lang | Task | knot-server/knot tokens | Read-the-code tokens | Reduction |
|------|------|------|------------:|---------------------:|----------:|
| spring-ai | Java | discovery — *how does the chat client run the advisor chain?* | 1 092 | 10 168 | **89.3%** |
| spring-ai | Java | callers — *who uses `ToolCallingManager`?* | 8 808 | 15 554 | **43.4%** |
| spring-ai | Java | explore — *structure of `DefaultChatClient.java`* | 4 865 | 7 838 | **37.9%** |
| puppeteer | TypeScript | discovery — *how is a CDP session created?* | 609 | 4 149 | **85.3%** |
| puppeteer | TypeScript | callers — *who calls `createCDPSession`?* | 1 004 | 39 878 | **97.5%** |
| puppeteer | TypeScript | explore — *structure of the `Page` API* | 7 287 | 25 300 | **71.2%** |
| knot | Rust | discovery — *how are call intents resolved?* | 594 | 14 824 | **96.0%** |
| knot | Rust | callers — *who calls `format_references_result`?* | 461 | 10 949 | **95.8%** |
| knot | Rust | explore — *structure of the graph query module* | 978 | 12 103 | **91.9%** |
| **TOTAL** | — | **9 tasks** | **25 698** | **140 763** | **81.7%** |

**≈ 5.5× fewer tokens** for the same nine questions — 115 000 tokens saved, enough to keep a long refactoring session inside a single context window.

<details>
<summary><b>Methodology (and how to reproduce it)</b></summary>

Both sides are measured on the **exact bytes an LLM would receive as tool output**, counted with OpenAI's `cl100k_base` tokenizer (tiktoken):

| Task | knot-server/knot side | Read-the-code side |
|------|-----------|--------------------|
| `discovery` | `GET /api/repos/{id}/search?q=<question>` | `rg -l <keyword>` (candidate list) **+ full read of the files that actually answer the question** |
| `callers` | `GET /api/repos/{id}/callers?entity=<symbol>` | `rg -n "\b<symbol>\b"` **+ full read of the first 5 distinct files with hits** |
| `explore` | `GET /api/repos/{id}/explore?path=<file>` | full read of the file |

The baseline is deliberately **generous**, so the measured saving is a lower bound:

- greps are restricted to the source files of the language (`-t java`, `-t ts`, `-t rust`) — no changelogs, no generated docs, no `node_modules`;
- for `discovery` the baseline is given *oracle file selection*: it reads only the files that answer the question, with zero wasted reads;
- for `callers` it reads at most 5 files, while a rigorous impact analysis would need every file with a textual hit.

Honest caveats: knot-server's cost scales with the **number of results**, not with repo size. The weakest row (`spring-ai` / `ToolCallingManager`, 43%) is a symbol with 156 references — knot-server/knot enumerates all of them with exact call sites, while the capped baseline reads only 5 files and still cannot tell a call from a comment. The `explore` rows for large classes are also the least favourable, because signatures plus docstrings are a large fraction of a well-documented file.

Repositories measured (as indexed): `spring-ai` 2 406 files / 25 733 entities, `puppeteer` 1 832 files / 19 310 entities, `knot` 222 files / 4 000 entities.

</details>

---

## ✨ Key Features & API Endpoints

**knot-server** provides a comprehensive REST API to manage the lifecycle of your codebases.

### 📦 Repository Management
- **`POST /api/repos`**: Register a new Git repository. Accepts a JSON body with a URL, name, and optional authentication. This endpoint is **idempotent**: if a repository with the same derived ID already exists, the server treats the call as a re-registration — the existing database entries and local files are cleaned up and the repository is cloned from scratch. The response message indicates whether the call was a fresh registration or a re-registration.
  ```json
  {
    "url": "https://github.com/raultov/knot.git",
    "name": "knot-core",
    "branch": "master",
    "webhook_secret": "your-secret-token",
    "auth": { "type": "none" }
  }
  ```
  | Field | Required | Description |
  |-------|----------|-------------|
  | `url` | Yes | Git repository URL (HTTPS, SSH, or local path) |
  | `name` | No | Display name (auto-derived from URL if omitted) |
  | `branch` | No | Branch to clone (defaults to `"main"`) |
  | `webhook_secret` | No | Shared secret for validating webhook signatures (HMAC-SHA256 or token). **Required to use the `/api/webhook` endpoint.** |
  | `auth` | No | Authentication method: `{"type": "ssh"}`, `{"type": "https", "token": "..."}`, or `{"type": "none"}` (default: `{"type": "ssh"}`) |

  > **Local filesystem paths in `url`:** When `url` is an absolute path to a directory
  > on the local filesystem (e.g. `/home/raul/workspace/my-app`), the server bypasses
  > `git clone` / `git fetch` and instead mirrors the source's working tree into the
  > workspace. This means **uncommitted working-tree changes in the source are
  > picked up by the next sync** — useful when indexing a repository you are
  > actively developing. A regular `git fetch` only transfers committed objects, so
  > it would miss in-flight edits. See the `Indexing local repositories` section
  > below for the recommended Docker setup.
  >
  > **Ignored artifact directories:** The local sync never copies the following
  > build / dependency / IDE directories (matched by base name anywhere in the
  > tree), and removes any that may have accumulated in the mirror from a
  > previous unfiltered sync:
  >
  > `target/`, `node_modules/`, `build/`, `dist/`, `out/`, `.gradle/`, `.next/`,
  > `.nuxt/`, `.svelte-kit/`, `.cache/`, `__pycache__/`, `.pytest_cache/`,
  > `.mypy_cache/`, `.ruff_cache/`, `.tox/`, `.idea/`, `.vscode/`
  >
   > This keeps the workspace small and keeps sync time bounded (a Rust project's
   > `target/` is routinely 10s of GB and would otherwise be mirrored on every
   > sync). The `.knot/` indexer-state directory and `.knot.lock` are never
   > copied from the source — the mirror's own incremental state is always
   > preserved.
- **`GET /api/repos`**: List all registered repositories, along with their current status (`pending`, `cloning`, `pulling`, `indexing`, `indexed`, `error`) and last indexed timestamp.
- **`GET /api/repos/:id`**: Retrieve detailed information about a specific repository.
- **`DELETE /api/repos/:id`**: Remove a repository from the registry and delete its local workspace. (No request body required).

### 🔄 Indexing & Webhooks
- **`POST /api/repos/:id/sync`**: Manually trigger an asynchronous sync and re-indexing job for a repository. (No request body required).
- **`GET /api/repos/:id/progress`**: Get live indexing progress (`percent_complete`, `stage`, parsed files, ingested entities). Note that progress reflects the latest pipeline run and resets when a new sync starts. If the server is restarted during indexing, progress may show as `idle` until the next scheduled sync auto-heals it.
- **`GET /api/progress`**: Batch progress for every registered repository in a single call. Each entry resolves via the in-process `ProgressTracker` first, then falls back to the on-disk snapshot at `<workspace>/progress/<id>.json` (written by whichever node is currently indexing that repo), so a request served by node B reports the real progress of a job running on node A. Ideal for refreshing a UI dropdown with one HTTP request.
- **`POST /api/webhook/:id`**: Endpoint for Git provider webhooks (GitHub, GitLab, Bitbucket). Securely validates payload signatures (HMAC-SHA256) or tokens, triggering a fast, incremental background re-index on push events. The request body should be the standard JSON webhook payload sent by the Git provider.

### 🔍 Code Intelligence Search
- **`GET /api/repos/:id/search?q=...&kinds=...&path=...&max_results=...`**: Semantic + structural search. Find code by meaning, class name, method signature, or docstrings. The optional `kinds` filter accepts comma-separated exact wire-format kinds (e.g. `rust_function`) or aliases like `definition`, `class`, `function` (knot's kind filter, forwarded verbatim). The optional `path` filter accepts a repo-relative directory prefix (`src/api`, matched on a path boundary) or a glob (`src/**/*_test.rs`). `max_results` is **enforced** at 1..=100 (default 5): requests above 100 are clamped to 100 — there is no pagination or cursor, so to look past the bound narrow the search with `kinds` / `path` or refine the query.
- **`GET /api/repos/:id/callers?entity=...&max_targets=...`**: Reverse dependency lookup. Identify callers, dead code, and perform impact analysis. Rows carry `repo_name` / `target_repo_name` (knot 1.8.1), so callers are self-labeling. The response always reports the true pre-truncation target count in `resolution.total_targets` and whether the relationship buckets are only a sample in `resolution.truncated`; `max_targets` (default 25, max 500) is the opt-in path to the full impact set.
- **`GET /api/repos/:id/explore?path=...`**: File anatomy inspection. Quickly see all classes, interfaces, methods, and functions in a specific file.
- **`GET /api/repos/:id/deps`**: View repository dependencies (transitive and reverse) across the indexed ecosystem.

The per-repo routes above remain **single-repo by design**. For queries spanning several
(or all) registered repositories, use the cross-repo routes below.

#### Cross-repo search & callers

- **`GET /api/search?q=...&repo=...&max_results=...&kinds=...&path=...`**: Semantic + structural search across
  one, several, or all registered repositories. Every result entity carries `repo_name`, so
  multi-repo results are self-labeling. The optional `kinds` filter (same syntax as the per-repo
  search) applies globally across the scope, and the optional `path` filter (same syntax as the
  per-repo search) applies within every repository of the scope.
- **`GET /api/callers?entity=...&repo=...&max_targets=...`**: Reverse dependency lookup across
  repositories. Every row identifies the repository of the caller (`repo_name`) and of the
  referenced entity (`target_repo_name`) — a genuine cross-repo reference is the row where the
  two differ. `resolution.targets[]` is labeled too, and `resolution.total_targets` /
  `resolution.truncated` make the completeness of the answer explicit.

Both routes share the same `repo` scope syntax:

| `repo` value | Meaning |
|--------------|---------|
| *(omitted)* | All **registered** repositories |
| `all` or `*` (case-insensitive) | All registered repositories (sentinel) |
| `repo-a` | Exactly one repository |
| `repo-a,repo-b` | Union of the listed repositories |
| *(any, empty registry)* | Empty result, `200` — the databases are not queried |

Caveats:

- `max_results` (search only, default 5, clamped to 1..=100 on both search routes, mirroring
  knot's MCP contract) is a **global** cap across the whole scope: with `repo=all` one dominant
  repository can crowd out the others. There is **no pagination or cursor** — when the bound is
  not enough, narrow the scope with `repo` / `kinds` / `path` or refine the query instead of
  raising the limit.
- `max_targets` (callers only, default 25, clamped to 1..=500) caps how many target entities
  the queried name is resolved against. Truncation is always explicit: `resolution.total_targets`
  is the true pre-truncation count and `resolution.truncated` is the flag; when it is `true`, the
  relationship buckets cover only `resolution.targets[]` (a sample), never the full impact set.
  Raise `max_targets` (up to 500) or pass a qualified name (`Namespace.Type.Member`) / narrow the
  scope for the complete set. Under `repo=all` a common name resolves against every registered
  repository, so the cap fills faster.
- A repository literally named `all` (or `*`) is not addressable through these routes (the
  token is the sentinel); use `/api/repos/all/search` and `/api/repos/all/callers`, which
  build a single-repo scope directly.
- Unknown repository ids are rejected with `400 Unknown repository ids: ...`. A registered
  but not-yet-indexed repo is a valid scope member that simply contributes no rows.
- `repo=all` — and an omitted `repo` — are confined to the registry: the sentinel expands
  to the registered repository ids, so rows from repositories that were deleted from the
  registry are never returned (previously the query ran unfiltered against the databases).
  With an empty registry both spellings return an empty result with `200` without querying.

### 🧬 Graph Visualization (Web UI)
- **`GET /graph`**: Interactive 3D codebase graph viewer. Open in your browser to visually explore entity relationships.
  - **Dynamic Filtering**: Real-time toggles for relationship types (`Calls`, `Extends`, `Implements`, `Overrides`, `Contains`, etc.) and entity kinds (`Classes`, `Interfaces`, `Functions`).
  - **Node Interaction**: 
    - **Click**: Automatically discover and expand neighbors.
    - **Focus on Entity**: Isolate a specific entity and its deep relationship subgraph.
    - **Back to Overview**: Return to the global entry-points view.
  - **High-Contrast Selection**: The currently selected node is highlighted in white for maximum visibility.
  - **Performance Optimized**: Default overview mode excludes noisy child relationships (`CONTAINS`), while focused mode uses physical hierarchy edges to maintain connectivity. Nested declarations (inner classes, C# nested records/enums) are included in the default overview.
  - **Smart Tooltips**: Hover over nodes to see Fully Qualified Names (FQN), kind, file path, and line numbers.
  - **Contextual Search**: Find entities by FQN or name; results include package/module context.
  - **Cross-Repository Search**: An "All repos" checkbox next to the search box switches it
    from the selected repository to every registered repository at once. Results are grouped
    and badged by repository, and clicking a result from another repository switches the
    active repository before focusing the entity. The 3D graph itself remains single-repo.
- **`GET /api/repos/:id/graph?entity=...`**: Query the entity subgraph for a given repository root entity. Returns nodes and edges in JSON format for programmatic consumption.

  | Parameter | Type | Default | Description |
  |-----------|------|---------|-------------|
  | `entity` | String | *optional* | Name or FQN of the root entity. If omitted, returns a repository overview (including nested declarations like inner classes, C# nested records/enums). |
  | `depth` | u32 | `2` | Traversal depth (1–5) |
  | `relationships` | CSV | `CALLS,EXTENDS,IMPLEMENTS` | Edge types to follow. |
  | `direction` | String | `both` | `outgoing`, `incoming`, or `both` |
  | `kinds` | CSV | `classes,interfaces` | Entity types to include. |

- **`GET /api/repos/:id/graph/repos`**: Query repository-level dependency graph (DEPENDS_ON relations). Returns nodes and edges in JSON format for repository-level cross-dependency tracking.

  | Parameter | Type | Default | Description |
  |-----------|------|---------|-------------|
  | `depth` | u32 | `3` | Traversal depth (1–5) |
  | `direction` | String | `both` | `outgoing`, `incoming`, or `both` |

  **Note on Repo-Deps View:** Enables 3D codebase visual tracking of dependencies/dependents across the indexed ecosystem in the web UI. Requires repositories with build manifests (such as Cargo.toml or package.json) to be indexed.

  **Response** (`200 OK`):
  ```json
  {
    "root_id": "app",
    "nodes": [
      { "id": "app", "name": "app", "build_system": "cargo", "group_id": "", "artifact_id": "app", "version": "1.0.0", "is_root": true, "registered": true, "relation": "root" }
    ],
    "edges": [
      { "source": "app", "target": "lib", "type": "DEPENDS_ON" }
    ],
    "total_nodes_found": 2
  }
  ```

  **Note on Overview Mode:** When no `entity` is provided, the server identifies "entry points" (entities not contained by others) and traverses from them using the selected relationship types. Disconnected nodes are automatically pruned in focused views.

  **Response** (`200 OK`):
  ```json
  {
    "root_id": "abc123...",
    "nodes": [
      { "id": "...", "name": "handleRequest", "kind": "rust_function", "language": "rust", "file_path": "src/handler.rs", "start_line": 42, "signature": "fn handleRequest(req: Request) -> Response" }
    ],
    "edges": [
      { "source": "...", "target": "...", "type": "CALLS" }
    ],
    "truncated": false,
    "total_nodes_found": 15
  }
  ```

- **`GET /api/repos/:id/graph/expand?entity=...&exclude=...`**: Same as `/graph` but with `depth=1` fixed, plus an `exclude` parameter (CSV of UUIDs) to skip nodes the frontend already has. Used by the graph viewer when clicking on unexpanded nodes.

### 📖 Interactive API Documentation (Swagger UI)

- **`GET /docs`**: Interactive Swagger UI page where you can browse every endpoint, inspect request/response schemas, and execute live "Try it out" requests — no external tools needed. The header shows the `knot-server` version, the linked `knot` library version, and the OpenAPI 3.1 stamp side by side, so you always know which versions produced the spec.
- **`GET /api-docs/openapi.json`**: Raw OpenAPI 3.1 JSON spec for importing into Postman, Insomnia, or generating client SDKs.
- **Postman Collection**: We also include a `knot-server.postman_collection.json` file in the repository root to help you quickly test the API with Postman.

The spec is auto-generated at compile time via [`utoipa`](https://crates.io/crates/utoipa) and embedded directly in the binary — no external CDN or internet access required.

### ⚙️ Cluster & Health
- **`GET /api/health`**: Check the health of the server, including connections to Qdrant and Neo4j, and view repository statistics.
- **Distributed Locking**: File-based locking (`.knot.lock`) allows multiple `knot-server` instances to share a single NFS/EFS workspace, ensuring only one instance indexes a given repository at a time.
- **Background Scheduler**: Automatically detects and cleans up stale locks, and periodically re-indexes repositories that haven't been synced recently.

### 🛠️ Development & Quality Gates

```bash
make check                                  # Run all local quality gates (fmt, clippy, test, dupes)

# Or run gates individually:
cargo clippy --all-targets -- -D warnings  # Must pass
cargo fmt -- --check                        # Must pass
cargo test --all-targets                    # Run unit tests
cargo dupes check                           # Code duplication check
```

---

## 🔌 MCP Endpoint (`/mcp`)

knot-server is also an **MCP server**. The `/mcp` endpoint speaks the
[Model Context Protocol](https://modelcontextprotocol.io) over stateless
JSON-RPC HTTP (`POST /mcp`), so MCP clients (Claude Code, opencode, Cursor, …)
can connect directly to knot-server — including a load-balanced cluster.

The endpoint serves the **exact same six tools** as the `knot-mcp` stdio
binary, backed by the same Neo4j and Qdrant connections the REST API uses:

| Tool | Purpose |
|------|---------|
| `search_hybrid_context` | Semantic + structural code search with dependencies |
| `find_callers` | Reverse dependency lookup (impact analysis) |
| `explore_file` | File structure and entity declarations |
| `list_files` | File layout discovery (repo-relative listing, optional prefix/glob matcher) |
| `list_repo_dependencies` | Cross-repository dependency graph traversal |
| `list_repositories` | List all indexed repositories with optional name filtering |

`search_hybrid_context` carries knot's result-bound contract verbatim: `max_results`
is 1..=100 (default 5) and is **enforced** — a larger request is clamped to 100 and
the reply says so. There is no pagination: when the bound is not enough, narrow the
search with `kinds` / `path` / `repo_name` or refine the query. The REST search routes
enforce the same default and ceiling (the bounds are derived from knot's constants, so
they cannot drift).

`find_callers` carries knot's truncation contract verbatim: `resolution.total_targets`
is the true pre-truncation target count, `resolution.truncated` flags when the buckets
are only a sample, and the `max_targets` argument (default 25, max 500) is the opt-in
path to the full impact set. Because `/mcp` is a faithful passthrough of knot, REST and
MCP report the **same total** for the same entity.

### What `/mcp` exposes vs. the REST API

The five MCP tools are the **read** surface. They mirror the `skills/*.md`
guide almost 1:1, with these differences:

| Capability | MCP tool | REST equivalent |
|------------|----------|-----------------|
| Semantic code search | `search_hybrid_context` | `GET /api/repos/{id}/search`, `GET /api/search` |
| Caller / impact analysis | `find_callers` | `GET /api/repos/{id}/callers`, `GET /api/callers` |
| File anatomy | `explore_file` | `GET /api/repos/{id}/explore` |
| File layout discovery | `list_files` | — **MCP only** (by design: it is an agent-oriented "what files exist here?" aid used to pick `path` filters; REST clients have `GET /api/repos/{id}/explore` and the search routes' `path` filter instead) |
| Cross-repo dependencies | `list_repo_dependencies` | `GET /api/repos/{id}/deps`, `GET /api/repos/{id}/graph/repos` |
| List indexed repositories | `list_repositories` | `GET /api/repos` |
| Register / sync / delete a repository | — **REST only** | `POST /api/repos`, `POST /api/repos/{id}/sync`, `DELETE /api/repos/{id}` |
| Server health, indexing progress | — **REST only** | `GET /api/health`, `GET /api/repos/{id}/progress` |
| Raw entity subgraph | — **REST only** | `GET /api/repos/{id}/graph` |

`/mcp` is read-only: if a repository is not indexed yet, register it through
the REST API (or ask the operator) before calling the tools. The `initialize`
response repeats this in its `instructions`, so a well-behaved client learns it
during the handshake.

### Client configuration

Point your MCP client at the server (or the cluster's load balancer). The
configuration syntax differs per tool — use the block that matches yours.

**opencode** (`opencode.json`):

```json
{
  "mcp": {
    "knot": {
      "type": "remote",
      "url": "http://localhost:3000/mcp",
      "enabled": true
    }
  }
}
```

**Claude Code** (CLI, or a project-scoped `.mcp.json`):

```bash
claude mcp add --transport http knot http://localhost:3000/mcp        # user scope
claude mcp add --transport http --scope project knot http://localhost:3000/mcp
```

The manual `.mcp.json` form is `{"mcpServers":{"knot":{"type":"http","url":"http://localhost:3000/mcp"}}}`.

**Codex CLI** (`~/.codex/config.toml`):

```toml
[mcp_servers.knot]
url = "http://localhost:3000/mcp"
```

Some Codex versions require the experimental Rust MCP client for remote HTTP;
add `[features]` / `experimental_use_rmcp_client = true` if the server does not
appear. Verify from inside a session with `/mcp`.

**Cursor** (`.cursor/mcp.json`):

```json
{ "mcpServers": { "knot": { "url": "http://localhost:3000/mcp" } } }
```

**VS Code / GitHub Copilot** (`.vscode/mcp.json` — note the `servers` key):

```json
{ "servers": { "knot": { "type": "http", "url": "http://localhost:3000/mcp" } } }
```

**Gemini CLI** (`~/.gemini/settings.json` — note `httpUrl`, not `url`):

```json
{ "mcpServers": { "knot": { "httpUrl": "http://localhost:3000/mcp" } } }
```

To smoke-test the endpoint without any client, `tools/list` works before any
`initialize` — that is what stateless means:

```bash
curl -s -X POST http://localhost:3000/mcp \
  -H 'Content-Type: application/json' -H 'Accept: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
  | jq '.result.tools[].name'
```

### Troubleshooting

The endpoint also appears in Swagger UI (`/docs` → **MCP**) with a ready-to-run
`tools/list` example, which is the fastest way to check connectivity, the
`KNOT_SERVER_MCP_ENABLED` flag and the tool surface.

| Symptom | Meaning |
|---------|---------|
| `GET /mcp` → `405` + `Allow: POST, DELETE` | **Correct.** A stateless server opens no SSE stream, so there is nothing to attach to. Compliant clients tolerate this. |
| `406 Not Acceptable` | The `Accept` header excludes `application/json` (and `*/*` / `type/*`). Use a JSON-capable client or set `Accept: application/json`. |
| `415 Unsupported Media Type` | `Content-Type` is not `application/json`. |
| `400` + `code -32600` | The body is a JSON-RPC **batch**; batching was removed from the protocol in 2025-06-18. Send one message per request. |
| `400` + `code -32700` | The body is not valid JSON. |
| `200` + `error.code -32601` | Unknown method (e.g. `resources/list`). The five tools live under `tools/*`. |
| `404` on `/mcp` | `KNOT_SERVER_MCP_ENABLED=false` (the route is not mounted). |

### Statelessness — no sticky sessions required

`/mcp` is **stateless by design**: the server never issues a
`Mcp-Session-Id` header, keeps no handshake state, and every request is
self-contained. A load balancer can distribute requests by load with **no
session affinity**, and rolling deployments need no session draining — any
node can answer any request, including `tools/list` and `tools/call` from a
node that never saw the client's `initialize`.

### Scope semantics (`repo_name`)

Over `/mcp`, `repo_name: "all"` means **everything indexed in Neo4j** — a
faithful passthrough to the knot engine, exactly as `knot-mcp` behaves. By
contrast, `repo=all` on the REST endpoints (`/api/search`, `/api/callers`)
expands to the **registered** repositories and rejects unknown names. If the
graph and the registry ever diverge (e.g. a repository was deleted from the
registry but not from Neo4j), the two surfaces report different corpora. This
is intentional — `/mcp` must be a faithful copy of `knot-mcp` — but it is
worth knowing when mixing transports.

### Configuration & security

- `KNOT_SERVER_MCP_ENABLED` (default `true`): set to `false` to unmount `/mcp`
  entirely (requests then hit the 404 fallback).
- `/mcp` is **unauthenticated**, like the rest of the API. Anyone who can
  reach it can read the whole index. Protect it at the network layer — or, if
  you build your own gateway, every client above supports a `headers` field
  so a bearer token can be attached at the edge.

---

## 🛠️ Installation

<details open>
<summary><b>For Humans</b></summary>

**Option A: Let an LLM do it**

Paste this into any LLM agent (Claude Code, OpenCode, Cursor, etc.):

```
Install knot-server via Docker Compose and then install the agent skills by following: https://raw.githubusercontent.com/raultov/knot-server/master/README.md
```

</details>

<details>
<summary><b>For LLM Agents</b></summary>

### Step-by-Step Instructions

Follow these two phases in order. Ask the user for input where indicated.

#### Phase 1: Docker Compose Installation

1. **Check prerequisites**: Verify Docker and Docker Compose are installed:
   ```bash
   docker --version && docker compose version
   ```
   If either is missing, stop and tell the user to install Docker first.

2. **Choose an install directory**: Ask the user where to install the Docker Compose files.
   Default: `~/knot-server`.
   ```bash
   mkdir -p ~/knot-server && cd ~/knot-server
   ```

3. **Download the required files**:
   ```bash
   curl -O https://raw.githubusercontent.com/raultov/knot-server/master/docker-compose.yml
   curl -O https://raw.githubusercontent.com/raultov/knot-server/master/.env.example
   curl -O https://raw.githubusercontent.com/raultov/knot-server/master/up.sh
   curl -O https://raw.githubusercontent.com/raultov/knot-server/master/down.sh
   chmod +x up.sh down.sh
   cp .env.example .env
   ```

4. **Configure `.env`**: Ask the user if they want to index local repositories.
   - If **yes**, ask for the parent directory path (e.g. `/home/user/workspace`) and set
     `KNOT_LOCAL_REPOS_DIR` in `.env`.
   - If **no**, leave the default.

5. **Create the placeholder directory** (required on first run):
   ```bash
   mkdir -p ~/.knot/empty
   ```

6. **Start the stack**:
   ```bash
   ./up.sh -d
   ```

7. **Verify the server is healthy** (wait up to 30 seconds for services to start):
   ```bash
   sleep 15 && curl -fsS http://localhost:3000/api/health | jq
   ```
   Expected: `"status": "ok"`. If it fails, retry after another 15 seconds.
   If it still fails, check `docker compose logs` for errors.

#### Phase 2: Agent Skills Installation

8. **Run the skills installer**:
   ```bash
   curl -fsSL https://raw.githubusercontent.com/raultov/knot-server/master/.knot-server-agent-skills.sh | bash
   ```
   This extracts 9 skill documentation files and prompts interactively to register
   them with your AI agent (OpenCode, Claude Code, Gemini CLI, or universal).

### Verification

After both phases, confirm everything works:

```bash
# Server health
curl -fsS http://localhost:3000/api/health | jq

# Open the Swagger UI (optional)
# http://localhost:3000/docs

# Open the graph viewer (optional)
# http://localhost:3000/graph
```

</details>

### Prerequisites

| Component    | Version | Notes                              |
|--------------|---------|-----------------------------------|
| Docker       | 20.10+  | For running Qdrant and Neo4j      |
| qdrant       | 1.x     | Vector database (docker)          |
| neo4j        | 5.x     | Graph database (docker)           |

### 🐳 Official Docker Image

The official Docker image is available on Docker Hub:
**[`raultov/knot-server:latest`](https://hub.docker.com/r/raultov/knot-server)**

This image is lightweight (`debian:trixie-slim` based) and comes pre-packaged
with the `knot-server` binary, `git`, and SSH clients — everything needed to
clone and index repositories. It is the recommended way to deploy `knot-server`
in containerized environments (Docker, Docker Compose, or Kubernetes).

### Option A: Quick Install (curl)

A single command that auto-detects your OS and architecture — no `sudo` or
manual platform selection needed:

```bash
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/raultov/knot-server/releases/latest/download/knot-server-installer.sh | sh
```

For a specific version, replace `latest` with the version tag:
```bash
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/raultov/knot-server/releases/latest/download/knot-server-installer.sh | sh
```

### Option B: Docker Compose (Pre-built Image)

The easiest way to run `knot-server` with its dependencies. Just download the
`docker-compose.yml` file and run:

```bash
curl -O https://raw.githubusercontent.com/raultov/knot-server/master/docker-compose.yml
curl -O https://raw.githubusercontent.com/raultov/knot-server/master/.env.example
curl -O https://raw.githubusercontent.com/raultov/knot-server/master/up.sh
curl -O https://raw.githubusercontent.com/raultov/knot-server/master/down.sh
chmod +x up.sh down.sh
cp .env.example .env          # edit .env to set KNOT_LOCAL_REPOS_DIR etc.
# Create the required empty placeholder directory (only needed once)
mkdir -p ~/.knot/empty
./up.sh -d
```

This pulls the pre-built [`raultov/knot-server`](https://hub.docker.com/r/raultov/knot-server)
image from Docker Hub along with Qdrant and Neo4j — no compilation needed.

The `up.sh` and `down.sh` scripts are convenience wrappers around `docker compose`.
See the [Convenience Scripts](#-convenience-scripts) section below for usage details.

#### SSH credentials for private repositories

The container copies your SSH keys at startup and fixes permissions automatically
(avoiding the `Bad owner or permissions` error that occurs with a direct bind-mount
into `/root/.ssh`).

By default it uses `~/.ssh`. Override with `KNOT_SSH_KEYS_DIR`:

```bash
# Use a specific key directory (e.g. corporate Bitbucket keys)
KNOT_SSH_KEYS_DIR=/path/to/your/ssh/keys docker compose up
```

##### Passphrase-protected keys (corporate / enterprise environments)

Copying SSH key files alone is not enough when keys are protected by a
passphrase — the `ssh-agent` running on the host must be forwarded into the
container. The `docker-compose.yml` does this automatically by mounting the
host socket:

```yaml
environment:
  - SSH_AUTH_SOCK=/ssh-agent
volumes:
  - ${SSH_AUTH_SOCK}:/ssh-agent:ro
```

Make sure your host `ssh-agent` is running and the key is loaded before
starting the stack (`ssh-add ~/.ssh/id_rsa`).

#### Indexing local repositories

To index a repository that lives on your host machine instead of a remote URL,
mount the parent directory into the container **at the same absolute path** so
that paths you pass to the API resolve transparently.

The easiest way is to set `KNOT_LOCAL_REPOS_DIR` in the `.env` file (copy
`.env.example` as a starting point):

```bash
# .env
KNOT_LOCAL_REPOS_DIR=/home/raultov/workspace
```

Then just run `docker compose up`. Alternatively, prefix the variable on the
command line:

```bash
KNOT_LOCAL_REPOS_DIR=/home/raultov/workspace docker compose up
```

Then register the repo with its local path:

```bash
curl -X POST http://localhost:3000/api/repos \
  -H "Content-Type: application/json" \
  -d '{
    "url": "/home/raultov/workspace/github/ui",
    "name": "ui",
    "branch": "master",
    "auth": { "type": "none" }
  }'
```

> **Note:** `KNOT_LOCAL_REPOS_DIR` is mounted **read-only**. The server will
> read the existing repo from that path — it skips `git clone` because `.git`
> already exists — and index it in place.

If you already have Neo4j and Qdrant running on your host machine (not in containers),
use `--network host` so the container can reach them via `localhost`:

```bash
docker run --network host \
  -v ${HOME}/.ssh:/tmp/ssh_keys:ro \
  raultov/knot-server:latest
```

> **Note:** The `raultov/knot-server` image does **not** include Neo4j or Qdrant.
> Running `docker run` without `--network host` and without pointing to external
> databases will fail — the container defaults to `localhost` which refers to
> itself, not your host.

### Option C: Build from Source

Clone the repository and build the binary:

```bash
git clone https://github.com/raultov/knot-server
cd knot-server
cargo build --release
```

#### Running with Docker Compose from source

The repo includes `docker-compose.dev.yml`, a development overlay that adds
`build: .` on top of `docker-compose.yml`. Use it when you want docker compose
to build the image locally instead of pulling from DockerHub:

```bash
# Build the image from source
docker compose -f docker-compose.yml -f docker-compose.dev.yml build

# Start the full stack using the locally built image
docker compose -f docker-compose.yml -f docker-compose.dev.yml up

# Rebuild and start in one step
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build

# With local repos exposed (see "Indexing local repositories" above)
KNOT_LOCAL_REPOS_DIR=/home/user/workspace \
  docker compose -f docker-compose.yml -f docker-compose.dev.yml up
```

> **Tip:** You can add a shell alias to avoid repeating the `-f` flags:
> ```bash
> alias dc-dev='docker compose -f docker-compose.yml -f docker-compose.dev.yml'
> dc-dev up --build
> ```

###  Convenience Scripts

The repo includes `up.sh` and `down.sh` wrappers around `docker compose`.

**Start the stack:**
```bash
# Default (port 3000)
./up.sh

# Detached mode
./up.sh -d

# With custom port
KNOT_SERVER_PORT=6060 ./up.sh -d

# Rebuild from source (dev overlay)
KNOT_SERVER_PORT=6060 docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d
```

**Stop the stack:**
```bash
# Stop containers, keep all data
./down.sh

# Stop and delete DB volumes (Qdrant, Neo4j, workspace)
./down.sh --clean

# Stop and delete only repos.json (keep DB volumes)
./down.sh --json

# Stop and delete EVERYTHING (volumes + repos.json)
./down.sh --all
```

| Flag | Effect |
|------|--------|
| *(none)* | Stop containers, preserve all data |
| `--clean` / `-c` | Also delete Docker volumes (Qdrant, Neo4j, workspace, fastembed cache) |
| `--json` / `-j` | Also delete `~/.knot/repos/repos.json` (repository registry) |
| `--all` / `-a` | Delete volumes AND repos.json |

---

## 🤖 Using with AI Assistants (Cursor, Copilot, Claude, Gemini)

`knot-server` transforms any LLM with terminal access (Cursor, GitHub Copilot,
Claude Code, Gemini CLI, opencode, Cline, Aider) into a codebase-aware engineer.
By teaching the LLM to call the REST API via `curl`, you give it **semantic
understanding** of your entire codebase — far beyond what `grep` or file embeddings
can provide.

The AI learns **five code intelligence skills** that replace traditional text search:

| # | Skill | Endpoint | Use Case |
|---|-------|----------|----------|
| 1 | **Semantic Search** | `/search?q=` | Find code by *meaning*, not exact text |
| 2 | **Callers Analysis** | `/callers?entity=` | Impact analysis — who uses this function? |
| 3 | **File Exploration** | `/explore?path=` | Get a file's structure without reading it |
| 4 | **Dependency Graph** | `/deps` | Cross-repo dependencies |
| 5 | **Graph Visualization** | `/graph` | Interactive 3D entity relationship explorer |
| 6 | **Index Repository** | `/index` | Register & index the current repo (OpenCode) |

These skills teach the LLM to **always prefer knot-server `curl` calls over
`grep`/`find`/`rg`** for code exploration, dramatically improving accuracy
and reducing hallucinations.

> **Prefer native MCP when your agent supports it.** If your assistant can talk
> MCP (opencode, Claude Code, Codex, Cursor, VS Code/Copilot, Gemini CLI), point
> it at the `/mcp` endpoint (see [MCP Endpoint](#-mcp-endpoint-mcp)) — it gets
> the five read tools without any `curl` plumbing. Keep these `curl` skills for
> the REST-only operations (register, sync, delete, health, progress, raw
> subgraphs) and for agents without MCP support.

### Install Agent Skills

Download the pre-built skill instructions to teach your AI agent how to use the `knot-server` REST API.

**Option 1: One-liner (Recommended)**
Downloads and runs the interactive installer, which extracts the 9 skills and prompts you to register them with your AI agent (OpenCode, Claude Code, Gemini CLI, or universal).

```bash
curl -fsSL https://raw.githubusercontent.com/raultov/knot-server/master/.knot-server-agent-skills.sh | bash
```

**Option 2: Local Script (for developers)**
If you cloned the repo, you can run the installer locally:
```bash
./scripts/install-agent-skills.sh
```
*(Note: If you edit the `skills/*.md` files, run `python3 scripts/generate_skills_script.py` to regenerate the `.knot-server-agent-skills.sh` bundle).*

**Option 3: Per-file Downloader**
If the tarball is blocked by your firewall, download the `.md` files individually:
```bash
curl -fsSL https://raw.githubusercontent.com/raultov/knot-server/master/scripts/download-agent-skills.sh | bash
```

### Manual Registration

If you skipped auto-registration or use a different AI tool (like Cursor or Copilot), point your tool's system prompt to the downloaded `.md` files.

**For Cursor** (`.cursorrules`):
```bash
echo "Read the agent skills in .knot-server-agent-skills/ and use the REST API." >> .cursorrules
```

**For GitHub Copilot** (`.github/copilot-instructions.md`):
```bash
mkdir -p .github && echo "Read the agent skills in .knot-server-agent-skills/ and use the REST API." >> .github/copilot-instructions.md
```

**For OpenCode** (`opencode.json` snippet if you skipped auto-registration):
```json
  "skills": {
    "knot-server-preflight": { "description": "MANDATORY STEP 0: Server health and index status check", "location": "file:///path/to/.knot-server-agent-skills/preflight.md" },
    "knot-server-search": { "description": "Use knot-server for semantic code discovery across indexed repositories", "location": "file:///path/to/.knot-server-agent-skills/search.md" },
    "knot-server-callers": { "description": "Use knot-server to find reverse dependencies and perform impact analysis", "location": "file:///path/to/.knot-server-agent-skills/callers.md" },
    "knot-server-explore": { "description": "Use knot-server to get a structural overview of a source file", "location": "file:///path/to/.knot-server-agent-skills/explore.md" },
    "knot-server-deps": { "description": "Use knot-server to traverse the repository dependency graph", "location": "file:///path/to/.knot-server-agent-skills/deps.md" },
    "knot-server-graph": { "description": "Use knot-server to query raw entity relationship subgraphs", "location": "file:///path/to/.knot-server-agent-skills/graph.md" },
    "knot-server-list-repos": { "description": "Use knot-server to list, register, sync, and delete repositories", "location": "file:///path/to/.knot-server-agent-skills/repos.md" },
    "knot-server-workflows": { "description": "Multi-step knot-server workflows: impact analysis, cross-repo exploration, refactoring patterns", "location": "file:///path/to/.knot-server-agent-skills/workflows.md" },
    "knot-server-index": { "description": "Register and index the current repository in knot-server", "location": "file:///path/to/.knot-server-agent-skills/index.md" }
  }
```

### The `/index` Command (OpenCode only)

OpenCode supports a `/index` command that registers the current repository in knot-server.
After installing with option 2 or 3 (OpenCode registration), the command is automatically
installed in `~/.config/opencode/commands/index.md`. You can then type `/index` in the
OpenCode chat to:

1. Check if knot-server is running
2. Register the current repository (if new) or trigger a re-index (if already registered)
3. Wait for indexing to complete
4. Verify the repo is queryable with a quick search

> **Note:** The `/index` command is supported in OpenCode (option 2/3), Claude Code
> (option 4), and Gemini CLI (option 5). For other tools (Cursor, Copilot, Codex),
> simply ask the agent: "Index this repository in knot-server" and it will use the
> `[[repos]]` skill to do it.

### How It Works

Each skill file injects a **system prompt** into the LLM that defines:

- **When** to use each endpoint (trigger phrases)
- **How** to construct the `curl` command (parameters, `jq` filters)
- **How** to interpret the JSON response (field meanings)

The LLM learns to:
1. Instead of `grep "authenticate"`, call `GET /api/repos/{id}/search?q=authentication+logic`
2. Instead of searching for callers manually, call `GET /api/repos/{id}/callers?entity=handleRequest`
3. Instead of `cat src/file.rs`, call `GET /api/repos/{id}/explore?path=src/file.rs` to get the outline first
4. Before breaking a shared library, call `GET /api/repos/{id}/deps` to see the impact

### Example: AI-Assisted Code Exploration

```
User: "Where is the password hashing logic?"
AI (via knot-server):
  curl "/api/repos/myproject/search?q=password+hashing" | jq
  → Found `hash_password` in `src/auth/crypto.rs:142`
  → Reads only lines 142-180 instead of entire file
```

---

## ⚙️ Configuration

`knot-server` is configured entirely via environment variables or CLI flags.

| Environment Variable | Default Value | Description |
|----------------------|---------------|-------------|
| `KNOT_SERVER_PORT` | `3000` | Port the REST API binds to |
| `KNOT_SERVER_BIND_ADDR` | `0.0.0.0` | Address the server binds to |
| `KNOT_WORKSPACE_DIR` | `/var/lib/knot/repos` | Directory where Git repos are cloned & locks are managed. Ensure the user running the server has write access (e.g., `export KNOT_WORKSPACE_DIR=$HOME/.knot/repos`). |
| `KNOT_SERVER_QDRANT_URL` | `http://localhost:6334` | URL to the Qdrant instance |
| `KNOT_SERVER_QDRANT_COLLECTION`| `knot_entities` | Qdrant collection name |
| `KNOT_SERVER_NEO4J_URI` | `bolt://localhost:7687` | URI to the Neo4j instance |
| `KNOT_SERVER_NEO4J_USER` | `neo4j` | Neo4j username |
| `KNOT_NEO4J_PASSWORD` | *(required)* | Neo4j password |
| `KNOT_SERVER_EMBED_DIM` | `384` | Embedding dimension. Validated at startup against the native dimension of `KNOT_EMBED_MODEL`; a mismatch aborts before any database is touched. |
| `KNOT_EMBED_MODEL` | `AllMiniLML6V2` | Embedding model used for **both** indexing and query embedding, owned by `knot`. One of `AllMiniLML6V2` (384), `BGESmallENV15` (384), `BGEBaseENV15` (768), `MultilingualE5Small` (384), `JinaEmbeddingsV2BaseCode` (768), `NomicEmbedTextV15` (768). Changing it requires a matching `KNOT_SERVER_EMBED_DIM` and a full re-index. |
| `KNOT_SERVER_RAYON_THREADS`| *(all cores)* | Number of threads for parallel source code parsing. Reduces CPU usage when set to a low value (e.g. `2`). |
| `KNOT_SERVER_BATCH_SIZE` | `64` | Number of code entities buffered in memory per indexing batch. Lower values reduce RAM usage. |
| `KNOT_SERVER_INGEST_CONCURRENCY` | `4` | Number of concurrent async tasks for embedding computation and database ingestion. Lower values reduce RAM and CPU usage. |
| `KNOT_SERVER_POLL_INTERVAL_SECS` | `86400` (24h) | How often the background scheduler runs |
| `KNOT_SERVER_MAX_INDEX_AGE_SECS` | `86400` (24h) | Age before a repository is automatically re-indexed |
| `KNOT_SERVER_STALE_LOCK_TIMEOUT_SECS` | `3600` (1h) | Timeout before a `.knot.lock` file is considered orphaned and removed |
| `KNOT_SERVER_QUEUE_CAPACITY` | `16` | Maximum number of jobs in the background indexing queue. Returns `429 Too Many Requests` when full. |
| `RUST_LOG` | `info` | Log level (`debug`, `info`, `warn`, `error`) |
| `KNOT_SERVER_METRICS_ENABLED` | `true` | Enable Prometheus metrics endpoint at `/metrics` |
| `KNOT_SERVER_MCP_ENABLED` | `true` | Enable the stateless MCP endpoint at `/mcp` |

> **Note:** When using Docker Compose, export `KNOT_SERVER_PORT` _before_ `docker compose up`
> so the port mapping in `docker-compose.yml` also changes (defaults to `3000:3000`).
> Example: `KNOT_SERVER_PORT=8080 docker compose up`

### Embedding Model Guard

Indexing and search must use the **same** embedding model: vectors built with one
model are not comparable with queries embedded by another, and because the
dimensions often match (e.g. two different 384-dim models) the mismatch would
not error — it would only silently degrade recall. `knot-server` therefore
resolves `KNOT_EMBED_MODEL` exactly as `knot` does and validates it against
`KNOT_SERVER_EMBED_DIM` at startup:

```
KNOT_EMBED_MODEL=BGEBaseENV15 KNOT_SERVER_EMBED_DIM=384 knot-server
# Error: KNOT_SERVER_EMBED_DIM (384) does not match the selected embedding model
#        'BGEBaseENV15' (native dimension 768). ...
```

Set the two consistently, then rebuild the index (`POST /api/repos/{id}/sync`
after a clean) whenever the model changes.

### Docker Compose Host Variables

These variables are consumed by `docker compose` itself (not by the server binary) to configure volume mounts.
Copy `.env.example` to `.env` and set the values there so you do not have to prefix them on every `docker compose up`.

| Variable | Default | Description |
|----------|---------|-------------|
| `KNOT_SSH_KEYS_DIR` | `~/.ssh` | Directory of SSH key *files* to make available inside the container. Keys are copied to `/root/.ssh` with correct ownership and permissions at startup. |
| `SSH_AUTH_SOCK` | *(host socket)* | Path to the host SSH agent socket. Forwarded into the container at `/ssh-agent` so passphrase-protected keys work without re-entering the passphrase. Requires the host `ssh-agent` to be running with the key already loaded (`ssh-add`). |
| `KNOT_LOCAL_REPOS_DIR` | `~/.knot/empty` | Host directory to mount at the same absolute path inside the container (read-only). Set this to the parent directory of any local repos you want to index by path. |

---

## 🎛️ Performance Tuning

`knot-server` is highly parallel by default, which can cause high CPU and memory
usage during indexing. Three environment variables control resource consumption:

| Variable | Controls | Default | Effect of lowering |
|----------|----------|---------|-------------------|
| `KNOT_SERVER_RAYON_THREADS` | CPU | all cores | Fewer parallel parsers → lower CPU, slightly slower |
| `KNOT_SERVER_BATCH_SIZE` | RAM | `64` | Fewer entities buffered in memory → lower RAM |
| `KNOT_SERVER_INGEST_CONCURRENCY` | RAM + CPU | `4` | Fewer concurrent embedding + DB writes → lower RAM and CPU |

### Preconfigured Profiles

| Profile | RAYON_THREADS | BATCH_SIZE | INGEST_CONCURRENCY | Expected RAM | Expected CPU |
|---------|---------------|------------|--------------------|--------------|--------------|
| **Kubernetes / Low memory** | `2` | `16` | `1` | < 1 GiB | ~200% |
| **Balanced** | `4` | `32` | `2` | ~2 GiB | ~400% |
| **Maximum throughput** (default) | all cores | `64` | `4` | ~5 GiB | all cores |

### Docker run with tuning

```bash
docker run --network host \
  -v ${HOME}/.ssh:/root/.ssh:ro \
  -e KNOT_SERVER_RAYON_THREADS=2 \
  -e KNOT_SERVER_BATCH_SIZE=16 \
  -e KNOT_SERVER_INGEST_CONCURRENCY=1 \
  raultov/knot-server:latest \
  --neo4j-password <your-password> \
  --workspace-dir /var/lib/knot/repos
```

### Docker Compose with tuning

```yaml
services:
  knot-server:
    image: raultov/knot-server:latest
    ports:
      - "3000:3000"
    environment:
      - KNOT_WORKSPACE_DIR=/var/lib/knot/repos
      - KNOT_SERVER_QDRANT_URL=http://qdrant:6334
      - KNOT_SERVER_NEO4J_URI=bolt://neo4j:7687
      - KNOT_SERVER_NEO4J_USER=neo4j
      - KNOT_NEO4J_PASSWORD=knotsecret
      - KNOT_SERVER_RAYON_THREADS=2
      - KNOT_SERVER_BATCH_SIZE=16
      - KNOT_SERVER_INGEST_CONCURRENCY=1
    volumes:
      - knot_workspace:/var/lib/knot/repos
    depends_on:
      qdrant:
        condition: service_started
      neo4j:
        condition: service_started
```

---

## 📊 Metrics

`knot-server` exposes Prometheus metrics at `GET /metrics` on the same port (default 3000). The endpoint requires no authentication and is served outside the OpenAPI spec (not visible in Swagger UI).

### Configuration

| Variable | Default | Description |
|----------|---------|-------------|
| `KNOT_SERVER_METRICS_ENABLED` | `true` | Enable/disable the Prometheus metrics endpoint |

### Prometheus Scrape Config

```yaml
scrape_configs:
  - job_name: 'knot-server'
    scrape_interval: 15s
    metrics_path: '/metrics'
    static_configs:
      - targets: ['knot-server:3000']
```

### Metric Reference

#### HTTP

| Metric | Type | Labels |
|--------|------|--------|
| `knot_http_requests_total` | counter | `route`, `method`, `status` |
| `knot_http_request_duration_seconds` | histogram | `route`, `method` |
| `knot_http_requests_in_flight` | gauge | — |

#### Indexing Pipeline

| Metric | Type | Labels |
|--------|------|--------|
| `knot_indexing_jobs_total` | counter | `repo_id`, `kind` (clone\|pull), `result` (ok\|err) |
| `knot_indexing_duration_seconds` | histogram | `kind`, `result` |
| `knot_indexing_percent_complete` | gauge | `repo_id`, `stage` |
| `knot_indexing_parsed_files` | gauge | `repo_id` |
| `knot_indexing_total_files` | gauge | `repo_id` |
| `knot_indexing_entities_ingested` | gauge | `repo_id` |
| `knot_indexing_last_success_timestamp_seconds` | gauge | `repo_id` |

#### Registry & Queue

| Metric | Type | Labels |
|--------|------|--------|
| `knot_repositories_total` | gauge | — |
| `knot_repositories_by_status` | gauge | `status` (pending, queued, indexed, cloning, pulling, indexing, error) |
| `knot_queue_available_capacity` | gauge | — |

#### Process

| Metric | Type | Labels |
|--------|------|--------|
| `knot_process_uptime_seconds` | gauge | — |
| `knot_build_info` | gauge (=1) | `version`, `knot_version` |

### Example Grafana Queries

| Panel | PromQL |
|-------|--------|
| Request rate per route | `sum by (route) (rate(knot_http_requests_total[1m]))` |
| 5xx error rate | `sum(rate(knot_http_requests_total{status=~"5.."}[5m]))` |
| Latency P95 | `histogram_quantile(0.95, sum by (le, route) (rate(knot_http_request_duration_seconds_bucket[5m])))` |
| Requests in flight | `knot_http_requests_in_flight` |
| Repos by status | `knot_repositories_by_status` |
| Queue capacity | `knot_queue_available_capacity` |
| Indexing duration P95 | `histogram_quantile(0.95, sum by (le, kind) (rate(knot_indexing_duration_seconds_bucket[15m])))` |
| Failed jobs per hour | `sum(increase(knot_indexing_jobs_total{result="err"}[1h]))` |
| Progress by repo | `knot_indexing_percent_complete` |
| Age of last index | `time() - knot_indexing_last_success_timestamp_seconds` |
| Uptime | `knot_process_uptime_seconds` |

### Security Note

`/metrics` has no authentication — it assumes deployment on an internal network. If port 3000 is exposed publicly, protect the endpoint via a reverse proxy (nginx, Traefik) rather than modifying the server.

---

## 🔭 Tracing (OpenTelemetry)

`knot-server` supports distributed tracing via OpenTelemetry, exporting spans to any OTLP gRPC compatible collector (Jaeger, Tempo, OpenTelemetry Collector, etc.).

### Configuration

| Variable | Default | Description |
|----------|---------|-------------|
| `KNOT_SERVER_TRACING_ENABLED` | `false` | Enable/disable OpenTelemetry tracing |
| `KNOT_SERVER_OTLP_ENDPOINT` | `http://localhost:4317` | OTLP gRPC endpoint URL |
| `KNOT_SERVER_TRACE_SAMPLE_RATIO` | `1.0` | Sampling ratio (0.0 to 1.0) |

When enabled, `knot-server` automatically instruments:
- All incoming HTTP requests (with `http.route` and status codes)
- Background worker jobs (git clone/pull, index pipeline)
- Scheduler poll loops
- W3C `traceparent` context propagation across service boundaries

---

Here is an end-to-end example of managing a repository with `knot-server` using `curl`:

**1. Start the server**
```bash
export KNOT_WORKSPACE_DIR=$HOME/.knot/repos
export KNOT_NEO4J_PASSWORD=mysecret
export KNOT_SERVER_QDRANT_URL=http://localhost:6334
export KNOT_SERVER_NEO4J_URI=bolt://localhost:7687
knot-server
```

**2. Register a repository**
```bash
curl -X POST http://localhost:3000/api/repos \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://github.com/raultov/knot.git",
    "name": "knot-core",
    "branch": "master",
    "webhook_secret": "my-webhook-secret"
  }'
```
*The server will instantly clone the repository and queue it for indexing.*

**3. Check indexing status**
```bash
curl http://localhost:3000/api/repos/knot-core
```
*Wait until `"status": "indexed"`.*

**4. Perform a semantic search**
```bash
curl "http://localhost:3000/api/repos/knot-core/search?q=webhook+validation"
```

**5. Trigger manual re-index (Sync)**
```bash
curl -X POST http://localhost:3000/api/repos/knot-core/sync
```

**6. Setup Git Webhooks**
In your GitHub/GitLab repository settings, add a webhook pointing to:
`http://your-server.com/api/webhook/knot-core`

Set the **secret/token** to the same value as `webhook_secret` you used when registering
the repository. Whenever a push occurs, `knot-server` will validate the signature and
automatically perform a fast incremental update.

**7. Browse the interactive API documentation**
Open `http://localhost:3000/docs` in your browser to explore all endpoints with Swagger UI. Use "Try it out" to test requests directly, or import `http://localhost:3000/api-docs/openapi.json` into Postman.

**8. Explore the codebase visually**
Open `http://localhost:3000/graph` in your browser. Select a repository from the
dropdown, search for an entity, and click nodes to expand their call/relationship graph
in 3D.

---

## 🚀 Cluster & High-Availability Deployment

`knot-server` is designed to run in horizontal scale-out clusters. Multiple instances
share a common workspace directory (NFS, EFS, or Kubernetes RWX PVC) and coordinate
via file-based locks — no distributed consensus protocol required.

### Docker Compose (Multi-Instance)

```yaml
services:
  knot-server:
    image: raultov/knot-server:latest
    environment:
      - KNOT_WORKSPACE_DIR=/var/lib/knot/repos
      - KNOT_SERVER_QDRANT_URL=http://qdrant:6334
      - KNOT_SERVER_NEO4J_URI=bolt://neo4j:7687
      - KNOT_SERVER_NEO4J_USER=neo4j
      - KNOT_NEO4J_PASSWORD=your-secure-password
      # Performance tuning (see Performance Tuning section)
      # - KNOT_SERVER_RAYON_THREADS=2
      # - KNOT_SERVER_BATCH_SIZE=16
      # - KNOT_SERVER_INGEST_CONCURRENCY=1
    volumes:
      - knot_shared_workspace:/var/lib/knot/repos
      - ~/.ssh:/root/.ssh:ro
    deploy:
      replicas: 3
    depends_on:
      - qdrant
      - neo4j

  qdrant:
    image: qdrant/qdrant:latest
    volumes:
      - qdrant_data:/qdrant/storage

  neo4j:
    image: neo4j:5
    environment:
      - NEO4J_AUTH=neo4j/your-secure-password
    volumes:
      - neo4j_data:/data

volumes:
  knot_shared_workspace:
    driver: local
  qdrant_data:
  neo4j_data:
```

### Kubernetes

You can deploy the official `raultov/knot-server:latest` image to Kubernetes
with a standard `Deployment`.

> **Reference:** The included [`docker-compose.yml`](docker-compose.yml) file is
> the canonical reference for configuring `knot-server`. It documents the exact
> environment variables, service dependencies (Qdrant + Neo4j), and volume
> mounts you need to translate into Kubernetes Deployments, Services, and
> ConfigMaps.

In Kubernetes, the key requirement for horizontal scaling is a
`PersistentVolumeClaim` with **`accessModes: [ReadWriteMany]`** (RWX). This
allows all knot-server Pods to share the workspace and coordinate safely.

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: knot-shared-workspace
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
  # storageClassName: nfs-client  # or efs-sc, cephfs, etc.
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: knot-server
spec:
  replicas: 3
  selector:
    matchLabels:
      app: knot-server
  template:
    metadata:
      labels:
        app: knot-server
    spec:
      containers:
        - name: knot-server
          image: raultov/knot-server:latest
          ports:
            - containerPort: 3000
          env:
            - name: KNOT_WORKSPACE_DIR
              value: /var/lib/knot/repos
            - name: KNOT_SERVER_QDRANT_URL
              value: http://qdrant.default.svc.cluster.local:6334
            - name: KNOT_SERVER_NEO4J_URI
              value: bolt://neo4j.default.svc.cluster.local:7687
            - name: KNOT_SERVER_NEO4J_USER
              value: neo4j
            - name: KNOT_NEO4J_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: knot-secrets
                  key: neo4j-password
            - name: KNOT_SERVER_RAYON_THREADS
              value: "2"
            - name: KNOT_SERVER_BATCH_SIZE
              value: "16"
            - name: KNOT_SERVER_INGEST_CONCURRENCY
              value: "1"
          resources:
            requests:
              memory: "512Mi"
              cpu: "500m"
            limits:
              memory: "1Gi"
              cpu: "2000m"
          volumeMounts:
            - name: shared-workspace
              mountPath: /var/lib/knot/repos
      volumes:
        - name: shared-workspace
          persistentVolumeClaim:
            claimName: knot-shared-workspace
```

Any Pod can receive webhook events or sync requests; the shared workspace
(`repos.json`, `.knot.lock` files) ensures exactly-once processing per repository.

---

## 🗺️ Roadmap

    - Language support in the knot library: Java, Kotlin, JavaScript, TypeScript, Rust, Python, and Varnish have been refined and are polished for production use. Groovy has received significant improvements in v1.5.6 (property accessor synthesis, parser/Javadoc hardening); further refinement is planned to complete verified coverage of the JVM family. C# has been recently added: knot-server categorises every C# declaration kind so C# repositories render correctly in the graph overview. C/C++ follows as the most widely used languages still pending deep verification.
- Implement language-based color coding in the `/graph` view to distinguish nodes by programming language.
- Resolve cross-file aliases for JavaScript and TypeScript (`require`, `import`): when a local alias shadows an imported entity, graph relationships should resolve to the original definition rather than the alias constant. Python alias resolution to follow. 
  See PR2 plan in the knot repository.
- After alias resolution: add a dedicated `TypeScriptModule` entity kind for synthetic `<module>` entities generated by re-export-only files, with its own color and filter toggle in `/graph`.
- Add a HELP section in the `/graph` viewer to assist users in understanding the graph visualization.

---

## 📜 License

This project is licensed under the **MIT License**. See [LICENSE](LICENSE) for details.