# knot-server
[](LICENSE)
[](https://www.rust-lang.org)
**knot-server** (v0.1.4) is a distributed REST API and background task scheduler for managing and indexing Git repositories across a cluster. It sits on top of the core [knot](https://github.com/raultov/knot) indexing engine, transforming it from a single-machine CLI tool into a highly available, cluster-aware enterprise service.
With `knot-server`, you can register Git repositories via a REST API, trigger automatic codebase indexing through webhooks (GitHub, GitLab, Bitbucket), and query the vector (Qdrant) and graph (Neo4j) databases—all while coordinating work safely across multiple server instances via NFS/EFS workspace locks.
---
## ✨ Key Features & API Endpoints
**knot-server** provides a comprehensive REST API to manage the lifecycle of your codebases.
### 📦 Repository Management
- **`POST /api/repos`**: Register a new Git repository. Accepts a JSON body with a URL, name, and optional authentication.
```json
{
"url": "https://github.com/raultov/knot.git",
"name": "knot-core",
"branch": "master",
"webhook_secret": "your-secret-token",
"auth": { "type": "none" }
}
```
| Field | Required | Description |
|-------|----------|-------------|
| `url` | Yes | Git repository URL (HTTPS, SSH, or local path) |
| `name` | No | Display name (auto-derived from URL if omitted) |
| `branch` | No | Branch to clone (defaults to `"main"`) |
| `webhook_secret` | No | Shared secret for validating webhook signatures (HMAC-SHA256 or token). **Required to use the `/api/webhook` endpoint.** |
| `auth` | No | Authentication method: `{"type": "ssh"}`, `{"type": "https", "token": "..."}`, or `{"type": "none"}` (default: `{"type": "ssh"}`) |
- **`GET /api/repos`**: List all registered repositories, along with their current status (`cloning`, `pulling`, `indexing`, `idle`, `error`) and last indexed timestamp.
- **`GET /api/repos/:id`**: Retrieve detailed information about a specific repository.
- **`DELETE /api/repos/:id`**: Remove a repository from the registry and delete its local workspace. (No request body required).
### 🔄 Indexing & Webhooks
- **`POST /api/repos/:id/sync`**: Manually trigger an asynchronous sync and re-indexing job for a repository. (No request body required).
- **`POST /api/webhook/:id`**: Endpoint for Git provider webhooks (GitHub, GitLab, Bitbucket). Securely validates payload signatures (HMAC-SHA256) or tokens, triggering a fast, incremental background re-index on push events. The request body should be the standard JSON webhook payload sent by the Git provider.
### 🔍 Code Intelligence Search
- **`GET /api/repos/:id/search?q=...`**: Semantic + structural search. Find code by meaning, class name, method signature, or docstrings.
- **`GET /api/repos/:id/callers?entity=...`**: Reverse dependency lookup. Identify callers, dead code, and perform impact analysis.
- **`GET /api/repos/:id/explore?path=...`**: File anatomy inspection. Quickly see all classes, interfaces, methods, and functions in a specific file.
- **`GET /api/repos/:id/deps`**: View repository dependencies (transitive and reverse) across the indexed ecosystem.
### ⚙️ Cluster & Health
- **`GET /api/health`**: Check the health of the server, including connections to Qdrant and Neo4j, and view repository statistics.
- **Distributed Locking**: File-based locking (`.knot.lock`) allows multiple `knot-server` instances to share a single NFS/EFS workspace, ensuring only one instance indexes a given repository at a time.
- **Background Scheduler**: Automatically detects and cleans up stale locks, and periodically re-indexes repositories that haven't been synced recently.
---
## 🤖 Using with AI Assistants (Cursor, Copilot, Claude, Gemini)
`knot-server` transforms any LLM with terminal access (Cursor, GitHub Copilot,
Claude Code, Gemini CLI, opencode, Cline, Aider) into a codebase-aware engineer.
By teaching the LLM to call the REST API via `curl`, you give it **semantic
understanding** of your entire codebase — far beyond what `grep` or file embeddings
can provide.
The AI learns **four code intelligence skills** that replace traditional text search:
| 1 | **Semantic Search** | `/search?q=` | Find code by *meaning*, not exact text |
| 2 | **Callers Analysis** | `/callers?entity=` | Impact analysis — who uses this function? |
| 3 | **File Exploration** | `/explore?path=` | Get a file's structure without reading it |
| 4 | **Dependency Graph** | `/deps` | Cross-repo dependencies |
These skills teach the LLM to **always prefer knot-server `curl` calls over
`grep`/`find`/`rg`** for code exploration, dramatically improving accuracy
and reducing hallucinations.
### Quick Install — One Command Per IDE
Download the pre-built skill instructions directly into your project:
**Cursor** (writes to `.cursorrules`):
```bash
curl -sL https://raw.githubusercontent.com/raultov/knot-server/master/skills/cursor-rules.md >> .cursorrules
```
**GitHub Copilot** (writes to `.github/copilot-instructions.md`):
```bash
mkdir -p .github && curl -sL https://raw.githubusercontent.com/raultov/knot-server/master/skills/copilot-instructions.md >> .github/copilot-instructions.md
```
**Claude Code / Gemini CLI / opencode / Cline / Aider** (generic system prompt):
```bash
curl -sL https://raw.githubusercontent.com/raultov/knot-server/master/skills/system-prompt.md > knot-skills.md
# Then instruct your agent: "Read knot-skills.md and use those tools"
```
### How It Works
Each skill file injects a **system prompt** into the LLM that defines:
- **When** to use each endpoint (trigger phrases)
- **How** to construct the `curl` command (parameters, `jq` filters)
- **How** to interpret the JSON response (field meanings)
The LLM learns to:
1. Instead of `grep "authenticate"`, call `GET /api/repos/{id}/search?q=authentication+logic`
2. Instead of searching for callers manually, call `GET /api/repos/{id}/callers?entity=handleRequest`
3. Instead of `cat src/file.rs`, call `GET /api/repos/{id}/explore?path=src/file.rs` to get the outline first
4. Before breaking a shared library, call `GET /api/repos/{id}/deps` to see the impact
### Example: AI-Assisted Code Exploration
```
User: "Where is the password hashing logic?"
AI (via knot-server):
→ Reads only lines 142-180 instead of entire file
```
---
## 🛠️ Installation
### Prerequisites
| Docker | 20.10+ | For running Qdrant and Neo4j |
| qdrant | 1.x | Vector database (docker) |
| neo4j | 5.x | Graph database (docker) |
### 🐳 Official Docker Image
The official Docker image is available on Docker Hub:
**[`raultov/knot-server:latest`](https://hub.docker.com/r/raultov/knot-server)**
This image is lightweight (`debian:trixie-slim` based) and comes pre-packaged
with the `knot-server` binary, `git`, and SSH clients — everything needed to
clone and index repositories. It is the recommended way to deploy `knot-server`
in containerized environments (Docker, Docker Compose, or Kubernetes).
### Option A: Quick Install (curl)
A single command that auto-detects your OS and architecture — no `sudo` or
manual platform selection needed:
```bash
For a specific version, replace `latest` with the version tag:
```bash
### Option B: Docker Compose (Pre-built Image)
The easiest way to run `knot-server` with its dependencies. Just download the
`docker-compose.yml` file and run:
```bash
curl -O https://raw.githubusercontent.com/raultov/knot-server/master/docker-compose.yml
docker compose up
```
This pulls the pre-built [`raultov/knot-server`](https://hub.docker.com/r/raultov/knot-server)
image from Docker Hub along with Qdrant and Neo4j — no compilation needed.
```yaml
services:
knot-server:
image: raultov/knot-server:latest
ports:
- "3000:3000"
environment:
- KNOT_WORKSPACE_DIR=/var/lib/knot/repos
- KNOT_SERVER_QDRANT_URL=http://qdrant:6334
- KNOT_SERVER_NEO4J_URI=bolt://neo4j:7687
- KNOT_SERVER_NEO4J_USER=neo4j
- KNOT_NEO4J_PASSWORD=knotsecret
volumes:
- knot_workspace:/var/lib/knot/repos
- ${HOME}/.ssh:/root/.ssh:ro
depends_on:
qdrant:
condition: service_started
neo4j:
condition: service_started
qdrant:
image: qdrant/qdrant:v1.16.2
volumes:
- qdrant_data:/qdrant/storage
neo4j:
image: neo4j:5.26-community
environment:
- NEO4J_AUTH=neo4j/knotsecret
- NEO4J_PLUGINS=["apoc"]
volumes:
- neo4j_data:/data
volumes:
knot_workspace:
qdrant_data:
neo4j_data:
```
### Option C: Build from Source
```bash
git clone https://github.com/raultov/knot-server
cd knot-server
cargo build --release
```
---
## ⚙️ Configuration
`knot-server` is configured entirely via environment variables or CLI flags.
| `KNOT_SERVER_PORT` | `3000` | Port the REST API binds to |
| `KNOT_SERVER_BIND_ADDR` | `0.0.0.0` | Address the server binds to |
| `KNOT_WORKSPACE_DIR` | `/var/lib/knot/repos` | Directory where Git repos are cloned & locks are managed. Ensure the user running the server has write access (e.g., `export KNOT_WORKSPACE_DIR=$HOME/.knot/repos`). |
| `KNOT_SERVER_QDRANT_URL` | `http://localhost:6334` | URL to the Qdrant instance |
| `KNOT_SERVER_QDRANT_COLLECTION`| `knot_entities` | Qdrant collection name |
| `KNOT_SERVER_NEO4J_URI` | `bolt://localhost:7687` | URI to the Neo4j instance |
| `KNOT_SERVER_NEO4J_USER` | `neo4j` | Neo4j username |
| `KNOT_NEO4J_PASSWORD` | *(required)* | Neo4j password |
| `KNOT_SERVER_EMBED_DIM` | `384` | Embedding dimension (must match the model) |
| `KNOT_SERVER_RAYON_THREADS`| *(logical cores - 1)* | Number of threads for parallel parsing |
| `KNOT_SERVER_POLL_INTERVAL_SECS` | `86400` (24h) | How often the background scheduler runs |
| `KNOT_SERVER_MAX_INDEX_AGE_SECS` | `86400` (24h) | Age before a repository is automatically re-indexed |
| `KNOT_SERVER_QUEUE_CAPACITY` | `16` | Maximum number of jobs in the background indexing queue. Returns `429 Too Many Requests` when full. |
| `RUST_LOG` | `info` | Log level (`debug`, `info`, `warn`, `error`) |
---
## 🔄 Example Workflow
Here is an end-to-end example of managing a repository with `knot-server` using `curl`:
**1. Start the server**
```bash
export KNOT_WORKSPACE_DIR=$HOME/.knot/repos
export KNOT_NEO4J_PASSWORD=mysecret
export KNOT_SERVER_QDRANT_URL=http://localhost:6334
export KNOT_SERVER_NEO4J_URI=bolt://localhost:7687
knot-server
```
**2. Register a repository**
```bash
curl -X POST http://localhost:3000/api/repos \
-H "Content-Type: application/json" \
-d '{
"url": "https://github.com/raultov/knot.git",
"name": "knot-core",
"branch": "master",
"webhook_secret": "my-webhook-secret"
}'
```
*The server will instantly clone the repository and queue it for indexing.*
**3. Check indexing status**
```bash
curl http://localhost:3000/api/repos/knot-core
```
*Wait until `"status": "idle"`.*
**4. Perform a semantic search**
```bash
curl "http://localhost:3000/api/repos/knot-core/search?q=webhook+validation"
```
**5. Trigger manual re-index (Sync)**
```bash
curl -X POST http://localhost:3000/api/repos/knot-core/sync
```
**6. Setup Git Webhooks**
In your GitHub/GitLab repository settings, add a webhook pointing to:
`http://your-server.com/api/webhook/knot-core`
Set the **secret/token** to the same value as `webhook_secret` you used when registering
the repository. Whenever a push occurs, `knot-server` will validate the signature and
automatically perform a fast incremental update.
---
## 🚀 Cluster & High-Availability Deployment
`knot-server` is designed to run in horizontal scale-out clusters. Multiple instances
share a common workspace directory (NFS, EFS, or Kubernetes RWX PVC) and coordinate
via file-based locks — no distributed consensus protocol required.
### Docker Compose (Multi-Instance)
```yaml
services:
knot-server:
image: raultov/knot-server:latest
environment:
- KNOT_WORKSPACE_DIR=/var/lib/knot/repos
- KNOT_SERVER_QDRANT_URL=http://qdrant:6334
- KNOT_SERVER_NEO4J_URI=bolt://neo4j:7687
- KNOT_SERVER_NEO4J_USER=neo4j
- KNOT_NEO4J_PASSWORD=your-secure-password
volumes:
- knot_shared_workspace:/var/lib/knot/repos
- ~/.ssh:/root/.ssh:ro
deploy:
replicas: 3
depends_on:
- qdrant
- neo4j
qdrant:
image: qdrant/qdrant:latest
volumes:
- qdrant_data:/qdrant/storage
neo4j:
image: neo4j:5
environment:
- NEO4J_AUTH=neo4j/your-secure-password
volumes:
- neo4j_data:/data
volumes:
knot_shared_workspace:
driver: local
qdrant_data:
neo4j_data:
```
### Kubernetes
You can deploy the official `raultov/knot-server:latest` image to Kubernetes
with a standard `Deployment`.
> **Reference:** The included [`docker-compose.yml`](docker-compose.yml) file is
> the canonical reference for configuring `knot-server`. It documents the exact
> environment variables, service dependencies (Qdrant + Neo4j), and volume
> mounts you need to translate into Kubernetes Deployments, Services, and
> ConfigMaps.
In Kubernetes, the key requirement for horizontal scaling is a
`PersistentVolumeClaim` with **`accessModes: [ReadWriteMany]`** (RWX). This
allows all knot-server Pods to share the workspace and coordinate safely.
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: knot-shared-workspace
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 50Gi
# storageClassName: nfs-client # or efs-sc, cephfs, etc.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: knot-server
spec:
replicas: 3
selector:
matchLabels:
app: knot-server
template:
metadata:
labels:
app: knot-server
spec:
containers:
- name: knot-server
image: raultov/knot-server:latest
ports:
- containerPort: 3000
env:
- name: KNOT_WORKSPACE_DIR
value: /var/lib/knot/repos
- name: KNOT_SERVER_QDRANT_URL
value: http://qdrant.default.svc.cluster.local:6334
- name: KNOT_SERVER_NEO4J_URI
value: bolt://neo4j.default.svc.cluster.local:7687
- name: KNOT_SERVER_NEO4J_USER
value: neo4j
- name: KNOT_NEO4J_PASSWORD
valueFrom:
secretKeyRef:
name: knot-secrets
key: neo4j-password
volumeMounts:
- name: shared-workspace
mountPath: /var/lib/knot/repos
volumes:
- name: shared-workspace
persistentVolumeClaim:
claimName: knot-shared-workspace
```
Any Pod can receive webhook events or sync requests; the shared workspace
(`repos.json`, `.knot.lock` files) ensures exactly-once processing per repository.
---
## 📜 License
This project is licensed under the **MIT License**. See [LICENSE](LICENSE) for details.