second-brain-cli 0.13.4

Memory layer for AI coding agents: ingest, recall, and sync across machines
# second-brain

Captures Claude Code conversation history automatically into a knowledge store with vector and lexical search, then serves it via MCP (Model Context Protocol) so any AI agent can recall what you've discussed, decided, and built across projects and machines.

## Memory Pipeline

1. **File watcher** detects new `.jsonl` in `~/.claude/projects/` (10s debounce)
2. **Hash check** skips unchanged files
3. **Adapter** parses tool-specific format into conversation chunks
4. **Heuristic filter** rejects noise: short or code-heavy text
5. **Dedup** checks first 128 chars against existing memories
6. **Embed** generates 384-dim BGE vector (local)
7. **Store** writes to SQLite with conversation and entity relations
8. **Consolidate** (manual) distills raw memories via LLM into concise knowledge facts, then cleans up raw

## Crates

| Crate | Description |
|-------|-------------|
| `second-brain-core` | SQLite storage, vector and lexical indexes, BGE-small-en-v1.5 embeddings, QueryEngine, schema types |
| `second-brain-adapters` | Parsers for Claude Code JSONL and Cursor JSONL formats |
| `second-brain-sync` | SSH transport, JSONL change log export/import, conflict resolution |
| `second-brain-mcp` | Thin MCP stdio server, proxies to daemon over HTTP |
| `second-brain-cli` | CLI binary for setup, ingest, recall, sync, consolidation |
| `second-brain-api` | HTTP daemon: owns DB, embedder, file watcher, serves all endpoints on port 7200 |

## Install

On Linux (x86_64 or aarch64, glibc 2.39 or newer, such as Ubuntu 24.04 or Debian 13) or an Apple Silicon Mac, with no Rust toolchain or GPU:

```bash
curl -fsSL https://raw.githubusercontent.com/Anyesh/second-brain/main/scripts/install.sh | sh
```

The script installs the latest release's binaries into `~/.cargo/bin` and runs `sb setup`. With a Rust toolchain you can instead run `cargo install second-brain-cli second-brain-api second-brain-mcp` and then `sb setup`.

`setup` handles everything: creates `~/.second-brain/`, installs a systemd user service (a launchd daemon on macOS), registers the MCP server with Claude Code (`claude mcp add --scope user`), adds hooks to Claude Code's settings (a Stop hook that asks each session that did real work for a note, recall matched to each prompt, and the project's open items at session and subagent start), starts the daemon, and runs the first ingest. If anything is missing afterwards, `sb doctor` names each failed check, why it failed and how to fix it.

Or build from source:

```bash
git clone https://github.com/Anyesh/second-brain.git
cd second-brain
cargo build --release
./target/release/sb setup
```

## Usage

### Ingest conversations

```bash
sb ingest
```

Ingest reads Claude Code history from `~/.claude/projects/` and Cursor history from `~/.cursor/projects/`, parses conversations, extracts memories, generates 384-dim embeddings, and stores everything in SQLite with entity extraction and relationship tracking.

### Recall

```bash
sb recall "how does authentication work in the API"
sb recall "what database did we choose and why"
```

Recall uses a weighted scoring function: 50% relevance (cross-encoder rerank score, or vector cosine when the reranker is off), 30% graph relevance (query entity mentions and relation types), 20% recency (exponential decay + access frequency).

### MCP server

`sb setup` registers this automatically. To register it by hand:

```bash
claude mcp add --scope user second-brain -e SECOND_BRAIN_API=http://127.0.0.1:7200 -- /path/to/second-brain-mcp
```

MCP tools available:

| Tool | Description |
|------|-------------|
| `recall` | Semantic search with graph-boosted scoring |
| `remember` | Store a new memory with embedding |
| `note` | Record a session's decision, dead ends, open items and what waits on the user |
| `ingest` | Trigger conversation ingestion |
| `associate` | Create explicit relations between memories |
| `forget` | Retire a memory by ID |
| `context` | Get a project's newest memories with their graph neighborhood |
| `status` | DB stats: memory count, index health |
| `list_projects` | List stored project paths with memory counts |

### Sync

Bidirectional sync between your devices over SSH using a sequence-numbered JSONL change log, sealed with your sync identity's key.

```bash
# Add a device: show a code on a device that is set up, then run the printed command on the new one
sb pair
sb pair <code> --host <host>
# ...or, if the new device already has memories it should drop (moved aside, readable for 90 days)
sb pair <code> --host <host> --discard-local

# Check a device: a fix for each failure, then per-peer sync state
sb doctor
sb sync-status

# Lose or retire a device: removes it, cuts its SSH access and rotates the store key
sb device list
sb device remove <name>                 # --keep <name> for devices it added that you still hold

# Copy the data dir while the daemon runs, for sb restore (the dest must not exist yet)
sb backup /mnt/backups/second-brain-<date>

# Rebuild a wiped device from a copy of a data dir
sb pair --restore-keys                  # on a surviving device: shows a code
sb restore <backup> --code <code> --host <host>
sb restore <backup> --own-keys          # the backup is this machine's own
sb restore <backup> < phrase.txt        # every device is gone: use the recovery phrase

# Recovery phrases: list, make one, retire one on every device
sb recovery
sb recovery new
sb recovery remove <fingerprint>

# Delete a memory, session or project on every device, then see which peers confirmed it
sb delete --memory <id>                 # or --session <id>, --project <path>
sb delete --status <delete id>
```

Each device encrypts its store, indexes and transcript archive at rest under its own keys, which are tied to its hardware: a data dir copied to another machine gets new keys there, so restore it with `sb restore`. Devices find each other on the local network by mDNS, so a changed IP does not break sync.

The first `sb pair` on a device with no sync identity starts its sync identity and prints a recovery phrase once; write it down, since with a copy of any device's data dir it restores your memory through `sb restore` after every device is lost. A new device with no memories of its own that can reach the device it paired with over SSH waits for its first sync from it before it ingests; `sb pair --stop-waiting` ends that wait.

The daemon syncs on its own with every device paired with `sb pair`, over SSH with keys and host keys that pairing installs. It also dials each line of an existing `~/.second-brain/sync-peers` (`user@host`, optionally followed by the path of `sb` on that peer when its non-interactive SSH PATH lacks it, which `sb sync` uses too unless given `--sb-path`; a host listed twice uses its first line) through your own SSH setup in batch mode, so that peer must accept a key without a prompt. A line that answers as a paired device becomes one of that device's routes, and once every line has, the daemon no longer reads the file. It runs a full pull and push at start and every five minutes, and pushes about five seconds after a local write; an unreachable peer is retried with a growing delay, up to 30 minutes, without holding up the others.

Sync resolves conflicts with last-write-wins on memory updates. Conversations and entities use create-or-skip semantics.

### Consolidation

Distill raw conversation memories into higher-level summaries using a local LLM.

```bash
sb consolidate                                        # default: 50 memories, concurrency 3
sb consolidate --limit 200                            # process more
sb consolidate --model gemma-4-E4B-it-Q4_K_M          # model (default: [llm].model with the [llm] endpoint, else this one)
sb consolidate --llm-url http://gpu-server:8081        # OpenAI-compatible endpoint (default: [llm].endpoint)
sb consolidate --concurrency 3                         # parallel LLM requests
sb consolidate --guard none                           # skip the identifier rewrite (default: retry)
sb consolidate --dry-run                               # preview without writing
```

Uses any OpenAI-compatible chat completions endpoint (llama.cpp, Ollama, vLLM), taken from `--llm-url` or the `[llm]` section of `~/.second-brain/config.toml`; with neither, `sb consolidate` exits with an error. When a distillate drops an exact identifier its excerpt names (a path, backticked command, ticket id or number), the default `--guard retry` asks the model once to rewrite it with them and keeps the rewrite only if it loses nothing the first draft had. After consolidation, raw memories are automatically cleaned up. Distilled memories are the primary knowledge store, linked to sources via `DistilledFrom` relations.

### Daily Pipeline

End-of-day automation: consolidate memories via local LLM.

```bash
sb daily-pipeline                            # consolidate memories
sb daily-pipeline --llm-url http://gpu-server:8081 --model my-model
```

**Requirements:**
- An OpenAI-compatible endpoint from `--llm-url` or `[llm]` in `~/.second-brain/config.toml`; without one the consolidate step is skipped
- Model: `--model`, else `[llm].model` when the endpoint also comes from `[llm]`, else `gemma-4-E4B-it-Q4_K_M` (so `--llm-url` without `--model` uses `gemma-4-E4B-it-Q4_K_M`)

### Memory Types

Memories are classified at ingest time (heuristic, no LLM):

| Type | Trigger patterns |
|------|-----------------|
| Task | TODO, FIXME, action item, remaining: |
| Decision | decided, chose, went with, agreed, selected |
| Architecture | architecture, data model, schema design, the approach is |
| Debugging | root cause, the bug, the issue is |
| Question | contains "?" + interrogative word |
| Procedural | steps:, to do this, run, install |
| Episodic | deployed, shipped, migrated, found that |
| Semantic | catch-all factual knowledge |

Priority order: first match wins (Task → Decision → ... → Semantic).

## Configuration

| Env var | Default | Description |
|---------|---------|-------------|
| `SECOND_BRAIN_DB` | `~/.second-brain/brain.db` | Path to the SQLite database |
| `SECOND_BRAIN_API` | `http://127.0.0.1:7200` | Daemon URL (used by MCP server and CLI) |
| `SECOND_BRAIN_BIND` | `127.0.0.1:7200` | Daemon listen address |
| `SECOND_BRAIN_RERANK_DOC_CHARS` | `1024` | Cross-encoder document character budget; `0` disables truncation |
| `SECOND_BRAIN_RERANK_POOL` | `min(cores, 4)` | Rerank session pool size (concurrency headroom, not throughput) |
| `SECOND_BRAIN_RERANK_CHECKOUT_SECS` | `30` | Seconds to wait for a free rerank session before falling back to the non-reranked hybrid path |
| `RUST_LOG` | - | Tracing filter (e.g. `info`, `second_brain_mcp=debug`) |

## Graph schema

Relation types:
- `relates_to` (Memory -> Memory): semantic relationship with strength
- `reinforces` (Memory -> Memory): one memory strengthens another
- `contradicts` (Memory -> Memory): conflicting information
- `supersedes` (Memory -> Memory): newer information replaces older
- `distilled_from` (Memory -> Memory): consolidation link
- `derived_from` (Memory -> Conversation): provenance tracking
- `mentions` (Memory -> Entity): entity extraction links

## Dependencies

- [fastembed](https://github.com/Anush008/fastembed-rs) 5 for BGE-small-en-v1.5 embeddings (384-dim, runs locally, no API calls)

## License

MIT