remem-ai 0.5.97

Persistent memory for Claude Code and OpenAI Codex coding agents
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
# remem: Persistent Memory for Claude Code and OpenAI Codex

> Open-source agent memory for Claude Code, OpenAI Codex, MCP, and long-running engineering work.

Language: **English** | [简体中文]README.zh-CN.md

`remem` is a single Rust binary that automatically captures, distills, and injects project context across Claude Code and OpenAI Codex sessions: decisions, patterns, preferences, and learnings. Stop re-explaining your project every new coding-agent session.

[![CI](https://github.com/majiayu000/remem/actions/workflows/ci.yml/badge.svg)](https://github.com/majiayu000/remem/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

![Remem Memory terminal demo](assets/remem-demo.gif)

## The Problem

- **Session amnesia**: every new Claude Code or Codex session starts from zero.
- **Lost context**: bug-fix rationale and design decisions disappear after the session ends.
- **Preference fatigue**: the same preferences must be repeated every session.
- **No continuity**: long-running work is hard to resume with confidence.

## How remem Solves This

| Without remem | With remem |
|---|---|
| "We use FTS5 trigram tokenizer..." (every session) | Injected automatically from memory |
| "Do not use `expect()` in non-test code" (again) | Preference surfaced before you ask |
| "Last session we decided to..." (reconstruct manually) | Decision history with rationale |
| Bug context lost after session ends | Root cause + fix preserved |

## Quickstart

```bash
brew install majiayu000/tap/remem
remem install --target codex
remem status
```

If you do not use Homebrew:

```bash
curl -fsSL https://raw.githubusercontent.com/majiayu000/remem/main/install.sh | env REMEM_NO_CONFIG=1 sh
~/.local/bin/remem install --target codex
~/.local/bin/remem status
```

`remem install --target codex` creates or updates:

- `~/.remem/.key` and the encrypted `~/.remem/remem.db`
- `~/.remem/config.toml` memory-AI profiles
- Codex MCP registration in `~/.codex/config.toml`
- Codex SessionStart/Stop hooks in `~/.codex/hooks.json`

Success looks like:

- `remem install` prints `key`, `db`, `config`, `MCP`, `hooks`, and `binary`
  lines.
- `remem status` prints database counts instead of an error.

Restart Codex after installation and finish one session. Then run `remem doctor`.
For a Codex-only setup, it reports Schema, Key format, Database, and the Codex
Hooks/MCP rows as ok. If Claude Code config directories already exist, Claude
rows can warn until you also run `remem install --target claude` or
`remem install --target all`. If it warns about multiple `remem` binaries,
follow the printed install-path fix so hooks keep using the intended binary.

For Claude Code, use `remem install --target claude`; to configure both hosts,
use `remem install --target all`.

## Other Install Channels

```bash
# Quick install options
curl -fsSL https://raw.githubusercontent.com/majiayu000/remem/main/install.sh | env REMEM_NO_CONFIG=1 REMEM_VERSION=vX.Y.Z sh
~/.local/bin/remem install --target codex

curl -fsSL https://raw.githubusercontent.com/majiayu000/remem/main/install.sh | env REMEM_NO_CONFIG=1 sh
~/.local/bin/remem install --target codex

curl -fsSL https://raw.githubusercontent.com/majiayu000/remem/main/install.sh | env REMEM_NO_CONFIG=1 REMEM_INSTALL_DIR=/usr/local/bin sh
remem install --target codex

# npm wrapper
npm install -g @remem-ai/remem
remem install --target codex

# Cargo
cargo install remem-ai --bin remem
remem install --target codex

# Manual GitHub Release download
curl -LO https://github.com/majiayu000/remem/releases/latest/download/remem-darwin-arm64.tar.gz
tar xzf remem-darwin-arm64.tar.gz
mv remem ~/.local/bin/
codesign -s - -f ~/.local/bin/remem  # required on macOS ARM
remem install --target codex

# Build from source
git clone https://github.com/majiayu000/remem.git
cd remem
cargo build --release
cp target/release/remem ~/.local/bin/
codesign -s - -f ~/.local/bin/remem  # required on macOS ARM
remem install --target codex
```

Use one canonical `remem` command on PATH. Standalone and source installs
should normally live at `~/.local/bin/remem`; Windows standalone installs
should use `%USERPROFILE%\.local\bin\remem.exe`. If you install through a
package manager such as Homebrew or Cargo, update through that same channel
and avoid keeping a second manual copy earlier or later on PATH. `remem doctor`
and `remem install --dry-run` warn when multiple `remem` executables are
visible.

### Updating an Existing Install

When you replace the binary manually, rerun `remem install` so existing Claude Code
and Codex hook commands pick up the current host-aware settings:

```bash
cargo build --release
cp target/release/remem ~/.local/bin/
codesign -s - -f ~/.local/bin/remem  # required on macOS ARM
remem install --target all
```

Verify the installed hooks include host-specific context commands:

```bash
jq -r '.hooks.SessionStart[]?.hooks[]?.command' ~/.claude/settings.json
jq -r '.hooks.SessionStart[]?.hooks[]?.command' ~/.codex/hooks.json
```

Expected commands are host-only; model, executor, and context policy live in
`~/.remem/config.toml`:

```text
/Users/you/.local/bin/remem context --host claude-code
/Users/you/.local/bin/remem context --host codex-cli
```

## Use With Codex

`remem install --target codex` configures Codex in four ways:

- Enables Codex hooks with `[features].hooks = true` in `~/.codex/config.toml`
- Registers `remem` as an MCP server in `~/.codex/config.toml`
- Writes Codex hook commands to `~/.codex/hooks.json`
- Creates or updates `~/.remem/config.toml` memory-AI profiles

After restarting Codex, remem automatically injects relevant project memory at
session start and summarizes the session at stop. Codex can also call the MCP
tools exposed by `remem mcp`, including `search`, `get_observations`,
`save_memory`, `workstreams`, and `timeline`.

The default Codex integration is intentionally low-noise: it uses
`SessionStart` for context injection and `Stop` for background summarization.
Codex uses strict duplicate-injection gating via
`[memory_ai.hosts."codex-cli"].context_gate = "strict"`, so a mid-chat
`SessionStart` repeat stays silent after the first injection for the same
session. It does not install high-frequency Bash observation by default.

### Codex Plugin

This repository includes a local Codex plugin wrapper in `plugins/remem`.
The plugin exposes `remem mcp` and a Remem skill while keeping hook activation
explicit. The complete product direction is documented in
[`docs/spec-codex-plugin-complete-design.md`](docs/spec-codex-plugin-complete-design.md);
the current plugin is the local development foundation, not the final
self-contained plugin experience. To try it from a local checkout:

```bash
codex plugin marketplace add .
codex plugin add remem@remem-local
```

After installing the plugin, start a new Codex thread. To enable automatic
SessionStart context injection and Stop summarization, run:

```bash
cargo build --release
node plugins/remem/scripts/activate-codex.js --dry-run
node plugins/remem/scripts/activate-codex.js
```

## Distribution Channels

Currently published:

- Homebrew: `brew install majiayu000/tap/remem`
- GitHub Releases: prebuilt binaries for macOS and Linux on x64/arm64
- crates.io: `cargo install remem-ai --bin remem`
- npm: `npm install -g @remem-ai/remem`
- Source build: `cargo build --release`

Good next channels:

- apt/yum packages: useful later, after the binary install path and service
  story are stable across Linux distributions

## How It Works

remem uses host-specific hook strategies:

```
Claude Code workflow
        |
        |- SessionStart      -> Inject memories + preferences
        |- UserPromptSubmit  -> Register session, flush stale queues
        |- PostToolUse       -> Capture tool operations (queued, <1ms)
        '- Stop              -> Summarize in background (~6ms return)

Codex workflow
        |
        |- SessionStart      -> Inject memories + preferences
        '- Stop              -> Summarize in background with Codex CLI
```

Codex does not install a high-frequency `PostToolUse(Bash)` observe hook by
default. Shell-heavy sessions must use the coalesced capture pipeline before
per-command capture is enabled again; otherwise Bash output can create an
unbounded backlog. Existing legacy hooks are also ignored unless
`REMEM_ENABLE_CODEX_BASH_OBSERVE=1` is set explicitly.

The capture pipeline starts with an append-only ledger:
`captured_events` stores raw hook/session evidence, `event_blobs` keeps large
payloads out of prompt-sized rows, and `extraction_tasks` coalesces work by
host/project/session instead of creating one LLM job per tool call. Curated
memory remains the promoted output of this pipeline, not the raw event itself.

## Remem vs Built-in `MEMORY.md`

Built-in memory files are enough when the context is small, stable, and worth
editing by hand: project rules, setup notes, and a short list of durable
preferences. Keep using them for facts that should be obvious at first glance.

Remem is meant for the parts that should not depend on manual upkeep:

- **Automatic capture and recall**: hooks summarize sessions into a SQLite
  memory store, while `remem search`, `remem show`, `timeline`, and MCP
  `get_observations` retrieve details on demand.
- **A bridge to native memory**: `remem sync-memory --cwd .` writes a compact
  `remem_sessions.md` entry for Claude Code native memory when that directory
  exists, with a `MEMORY.md` pointer and a size guard. Full detail stays in the
  database and is fetched with `remem search`.
- **A human-editable mirror**: `remem export --markdown --output
  ./remem-memory --project "$PWD"` writes one `.md` file per curated memory to
  an empty directory. After editing those files, `remem import markdown --source
  ./remem-memory` updates existing rows and rebuilds search, entity, embedding,
  and current-state indexes. Export refuses non-empty directories to avoid
  overwriting manual edits.
- **Governance and auditability**: `remem why <id>`, `remem govern --action
  stale --dry-run --json <id>`, `remem status --json`, and `remem usage --days
  14 --weeks 8` show why a memory is visible, what would change, store health,
  and memory-AI token/cost accounting.
- **Deterministic checks before claims**: local gates include
  `cargo test -q context::claude_memory --lib`, `cargo test -q eval::golden
  --lib`, `cargo test -q eval::governance --lib`, and `remem eval-e2e --json`.

Do not read this as a published claim that remem beats a carefully maintained
`MEMORY.md` on coding tasks. The flagship no-memory / remem / curated-file A/B
is still a separate benchmark requirement; until it is published, the honest
claim is capability coverage and reproducible local checks.

## Search Architecture

remem uses 4-channel Reciprocal Rank Fusion (RRF) inspired by [Hindsight](https://github.com/vectorize-io/hindsight):

```
Query: "database encryption"
        |
   +----+------------------------------------+
   |          4 parallel channels            |
   +-----------------------------------------+
   | 1. FTS5 (BM25)   trigram + OR           |
   | 2. Entity Index  1600+ entities         |
   | 3. Temporal      "yesterday"/"last week" |
   | 4. LIKE fallback short tokens           |
   +-------------+---------------------------+
                 |
        RRF score = sum(1 / (60 + rank_i))
                 |
             Top-K merged results
```

Enhancements:

- Entity graph expansion (2-hop multi-hop retrieval)
- Project-scoped entity search (no cross-project leakage)
- CJK segmentation support
- Chinese-English synonym expansion
- Title-weighted BM25 (`bm25(fts, 10.0, 1.0)`)
- Content-hash deduplication via `topic_key`
- Multi-step retrieval guidance in MCP tool descriptions

## Benchmark Snapshot

### LoCoMo (Informational Only)

Full [LoCoMo](https://github.com/snap-research/locomo) benchmark (10 conversations, 1540 QA pairs after adversarial skip):

This snapshot is a historical footnote and is not a CI or release gate. Use the
golden retrieval eval for deterministic gating; LoCoMo remains useful only for
manual, informational comparison because the methodology is disputed.

| Config | Overall | Single-hop | Multi-hop | Temporal | Open-domain | Ingest | Model |
|---|---:|---:|---:|---:|---:|---|---|
| **v1 (fair)** | **56.8%** | 67.1% | 39.0% | 53.9% | 28.1% | per-turn | gpt-5.4 |
| **v2 (optimized)** | **62.7%** | 72.3% | 61.3% | 40.5% | 56.2% | session_summary | gpt-5.4 |

### Internal Eval (1777 real memories)

| Metric | Value |
|---|---:|
| MRR | 0.858 |
| Hit Rate@5 | 1.000 |
| Dedup rate | 1.0% |
| Project leak | 0% |
| Self-retrieval | 100% |

### Local QA Eval

```bash
python3 eval/local/run_local_eval.py --db ~/.remem/remem.db --n 20
```

| Metric | Score |
|---|---:|
| Overall | **85.0%** |
| Decision | 77.8% |
| Discovery | 87.5% |
| Preference | 100% |
| Source in top-20 | 90.0% |

Requires explicit `--db` plus `.env` with `OPENAI_API_KEY` (optional `OPENAI_BASE_URL`, `OPENAI_MODEL`).

### Sandboxed E2E Eval

```bash
remem eval-e2e
remem eval-e2e --json
```

Runs a deterministic coding-agent memory corpus through the real local REST API
boundary (`POST /api/v1/memories`, then `GET /api/v1/search`) with a temporary
`REMEM_DATA_DIR`. The default run removes the sandbox directory afterward, so it
does not touch `~/.remem` or other real memory data. Use `--keep-data-dir` when
you need to inspect the generated database.

## Token Usage And Cost Reporting

remem records an AI usage ledger for its own background extraction, summary,
compression, and promotion calls. The CLI can report daily and weekly token
usage and estimated cost:

```bash
remem usage --days 14 --weeks 8
remem usage --project /path/to/project --days 30 --weeks 12
```

The report includes calls, input tokens, cache tokens, output tokens, reasoning
tokens, total tokens, estimated USD cost, and a precision note. Usage rows are
tagged by source:

- `anthropic_usage`: provider-reported usage from the Anthropic Messages API
- `codex_log`: exact token counts parsed from the current `codex exec --json`
  `turn.completed.usage` event
- `text_estimate`: fallback estimate from prompt/response text length

Cost is an estimate, not an invoice. Historical rows may be text estimates or
may have been repriced from older rows that did not store the exact model.

## Memory AI Configuration

Memory AI execution is configured in `~/.remem/config.toml` (override path with
`REMEM_CONFIG`). Hooks pass only `--host`; the config maps each host to one
profile used by summarize, flush/extract, compress, and dream.

```bash
remem config path
remem config show
remem config set memory_ai.profiles.codex.model gpt-5.2
```

For normal model switching, prefer the higher-level `remem model` commands:

```bash
remem model current
remem model list
remem model use cheap
remem model use balanced --dry-run
remem model use gpt-5.2 --reasoning medium
remem model use haiku --host claude-code
remem model test
remem model test --live
remem model rollback
```

`remem model test` only validates the selected config unless `--live` is set.
`remem model use` saves a rollback backup before writing the config. Built-in
presets are Codex-focused; use explicit model names for Claude Code profiles.

Default Codex profile:

```toml
[memory_ai.hosts."codex-cli"]
memory_profile = "codex"
context_gate = "strict"
context_color = true
capture_adapter = "codex-cli"

[memory_ai.profiles.codex]
executor = "codex-cli"
model = "gpt-5.2"
path = "codex"
```

## Commands

```bash
remem install
remem uninstall
remem doctor
remem search "query"
remem search "query" --branch main --type decision --multi-hop --offset 10
remem search "query" --json
remem show <id>
remem show <id> --json
remem eval
remem eval-e2e --json
remem eval-local
remem backfill-entities
remem encrypt
remem api --port 5567
remem status
remem status --json
remem config show
remem config set memory_ai.profiles.codex.model gpt-5.2
remem model current
remem model list
remem model use balanced --dry-run
remem model use gpt-5.2 --reasoning medium
remem model use haiku --host claude-code
remem model test [--live]
remem model rollback
remem usage --days 14 --weeks 8
remem pending list-failed
remem pending list-failed --json
remem pending retry-failed --dry-run
remem pending purge-failed --dry-run --older-than-days 7
remem govern --action stale --dry-run --json <id>
remem review list
remem review approve <id>
remem review discard <id>
remem review edit <id> --text "updated memory"
remem preferences list
remem preferences add "text"
remem preferences remove 42
remem context --cwd .
remem cleanup --dry-run --json
remem cleanup
remem dream [--project X] [--profile NAME] [--dry-run]
remem install --target codex
remem mcp
remem sync-memory --cwd .
```

### Scriptable JSON output

These commands emit one JSON object and no human text on stdout when `--json`
is set:

| Command | Stable top-level fields |
|---|---|
| `remem status --json` | `version`, `database`, `totals`, `capture_pipeline`, `pending_observations`, `jobs`, `worker_daemon`, `today`, `top_projects` |
| `remem cleanup --dry-run --json` | `dry_run`, `retention_days`, `plan`, `applied` |
| `remem search ... --json` | `query`, `project`, `memory_type`, `limit`, `offset`, `branch`, `include_stale`, `multi_hop_requested`, `explain_requested`, `count`, `has_more`, `next_offset`, `results`, `raw_hits`, `multi_hop`, `explain_details` |
| `remem show <id> --json` | `found`, `id`, `memory` |
| `remem pending list-failed --json` | `project`, `limit`, `count`, `failed` |
| `remem govern ... --json` | `dry_run`, `action`, `reason`, `affected` |

## REST API

```bash
remem api --port 5567
TOKEN=$(cat ~/.remem/.api-token)
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:5567/api/v1/status
```

Library users who build the router directly should call
`remem::api::ensure_api_token()` before `remem::api::build_router(...)`.

| Endpoint | Method | Description |
|---|---|---|
| `/api/v1/search?query=&project=&type=&limit=&offset=&branch=&multi_hop=` | GET | Search memories |
| `/api/v1/memory?id=` | GET | Get one memory |
| `/api/v1/memories` | POST | Save memory |
| `/api/v1/status` | GET | System status |

## Security

- SQLCipher encryption at rest (`remem encrypt`)
- Data directory permissions (`0700`)
- Key file permissions (`0600`)
- REST API binds localhost only (`127.0.0.1`) and requires
  `Authorization: Bearer $(cat ~/.remem/.api-token)`
- API token file permissions (`0600`)

## Architecture Docs

See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for full internals and data flow.

## Uninstall

```bash
remem uninstall
rm -rf ~/.remem
```

## License

MIT