engram-memory 0.6.0

Core binary and shared infrastructure for Engram memory system
Documentation
# Engram — Agent Instructions

Engram is a memory system for AI agents. Use it to store decisions, patterns, and bugfixes, then search them to inform future work.

## MCP Tools

### Core Operations

**memory_store** — Save a new memory.
```json
{
  "context": "Situation where the action occurred",
  "action": "Decision or action taken",
  "result": "Outcome or result",
  "memory_type": "decision",
  "tags": "auth,refactor",
  "project": "my-project"
}
```
**Выбор memory_type:**

| Тип | Когда использовать | Пример |
|-----|-------------------|--------|
| `decision` | Архитектурный/дизайн-выбор с обоснованием | "Выбрали REST вместо GraphQL потому что..." |
| `pattern` | Переиспользуемое решение, применимое к будущим задачам | "Svelte 5 stores через .svelte.ts с $state runes" |
| `bugfix` | Диагностика и исправление конкретного бага | "Токены истекали молча → добавили middleware" |
| `context` | Факт о проекте, статус фазы, конфигурация | "Phase B завершена, 124 теста, 17 файлов" |
| `antipattern` | Что НЕ делать и почему | "Не мокать БД в интеграционных тестах — прод миграция сломалась" |

**memory_search** — Find relevant memories using hybrid vector + sparse search.
```json
{
  "query": "authentication middleware architecture",
  "limit": 10,
  "project": "my-project",
  "tags": ["auth", "middleware"]
}
```
Optional `tags` (array of strings) filters results to memories that carry **all** specified tags.

**memory_judge** — Rate a memory's quality. Records the memory's score; the score is used by consolidation (the `min_score` stale-pruning filter). It does not influence live search ranking, which is positional RRF. The router's contextual-bandit Q-update is scaffolding and is not invoked during serving.
```json
{
  "memory_id": "uuid",
  "query": "the query that found this memory",
  "score": 0.8
}
```

**memory_delete** — Delete a memory by ID. Removes the memory and its HNSW index entry; FK-safe (clears feedback tracking rows first). Returns `NotFound` if the ID is absent.
```json
{
  "id": "uuid"
}
```

**memory_status** — Get system health: memory count, index size, pending judgments.
```json
{}
```

**memory_config** — Read current configuration.
```json
{
  "action": "get"
}
```

### Import / Export

**memory_export** — Export all active memories as portable JSON (excludes embeddings).
```json
{}
```

**memory_import** — Import memories from exported JSON. Merge mode: skips duplicates by ID.
```json
{
  "version": 1,
  "memories": [...]
}
```

### Consolidation

**memory_consolidate_preview** — Find deduplication candidates without changes.
```json
{
  "stale_days": 30,
  "min_score": 0.3
}
```
Each duplicate group reports `match_type`: `"exact"` (byte-identical
context/action/result triplets) or `"fts"` (BM25 full-text probing). The
`similarity` scale depends on `match_type`: exact groups carry the sentinel
`1.0`; FTS groups carry the mean |bm25| rank of the group members
(non-negative, unbounded above — NOT cosine). `primary_id` is the
deterministic group representative (exact: lexicographically smallest id;
FTS: the probe memory), not necessarily the merge survivor — analyze picks
the survivor independently. The `consolidation.fts_similarity_floor` config
(default `0.0` = disabled) drops FTS groups whose similarity falls below the
floor; exact groups are never filtered.

**memory_consolidate** — Analyze candidates with LLM. Returns merge/keep recommendations.
```json
{
  "stale_days": 30,
  "min_score": 0.3
}
```
Error contract: previously any LLM failure hard-failed the whole call. Now
the call succeeds and returns an `"errors"` array with one entry per failed
member or unloadable group; failed members produce no recommendation and do
not count toward `analyzed_count`. Note the apply-side consequence: during a
total LLM outage `memory_consolidate_apply` still executes stale/garbage
recommendations and the recommendations of groups whose members succeeded.

**memory_consolidate_apply** — Apply recommendations: merge, delete, or archive.
`min_confidence` (0.0–1.0, default 0.0 = apply all) skips recommendations below
that confidence — e.g. `0.7` applies only high-confidence merges. The CLI
equivalent is `engram consolidate apply --min-confidence 0.7`. The response
`"errors"` array merges analysis errors (prefixed `analyze ...`) with apply
errors.
```json
{
  "stale_days": 30,
  "min_score": 0.3,
  "min_confidence": 0.7
}
```

**memory_consolidate_log** — List consolidation history (merge/delete/archive audit
trail), newest first. `limit` (default 50) caps the number of entries.
```json
{
  "limit": 50
}
```

### Insights

**memory_insights** — List, generate, or delete derived knowledge.
```json
{
  "action": "list"
}
```
Actions: `list` (show insights), `generate` (run trainer analysis), `delete` (remove by ID with `"id": "uuid"`).

## Usage Patterns

### Store after completing work

After solving a bug, making an architecture decision, or discovering a pattern:

```
memory_store({
  context: "User auth tokens expired silently, no error in logs",
  action: "Added explicit token validation middleware with structured error logging",
  result: "Auth failures now surface immediately with clear error codes",
  memory_type: "bugfix",
  tags: "auth,middleware,logging",
  project: "api-server"
})
```

### Search before starting work

Before implementing a feature or debugging:

```
memory_search({ query: "authentication middleware patterns", limit: 5 })
```

### Feedback loop (IMPORTANT)

Engram uses two-phase feedback to learn which memories are useful:

**Phase 1 (automatic):** Every `memory_search` result is tracked as "shown to agent". No action needed.

**Phase 2 (your responsibility):** After using search results, call `memory_judge` for memories that actually helped:

```
memory_judge({ memory_id: "abc-123", query: "auth middleware", score: 0.9 })
```

**Scoring guide:**
- `0.8-1.0` — directly solved the problem or informed the decision
- `0.5-0.7` — useful context, partially relevant
- `0.1-0.4` — tangentially related, not very helpful
- Skip judge for irrelevant results (implicit low signal from no judge)

**When to judge:** after completing a task where search results contributed. Judge 1-3 most helpful memories, not all results.

Judging records the memory's score, consumed by consolidation's `min_score` filter. It does not affect live ranking, which is positional RRF; the trainer's `ranking_model.onnx` consumes score but is trained-but-unapplied scaffolding. The router's contextual-bandit Q-update (`Q ← Q + α(reward − Q)`, an EMA with no state transition and no discounted-future term, so not full reinforcement learning) is scaffolding and is not invoked during serving — searches use static per-mode defaults, and each search's decision is written to the `routing_log` table for offline analysis.

### Consolidation workflow

Periodically clean up duplicate and stale memories:

```
memory_consolidate_preview({})           → see candidates
memory_consolidate({})                   → get LLM recommendations
memory_consolidate_apply({})             → apply merges/deletes
memory_consolidate_log({})               → review the merge/delete audit trail
```

## Search Behavior

- **Hybrid search**: 70% vector similarity (HNSW cosine) + 30% sparse match (BM25 via FTS5)
- **HyDE**: opt-in via `embedding.hyde_threshold > 0` (disabled by default). When enabled, an LLM generates a hypothetical memory for queries shorter than the threshold and embeds the hypothesis instead of the raw query. Cache is keyed by the original query, so repeated identical queries hit the cache without an extra LLM call.
- **Cross-project**: searches current project first, applies score multiplier for other projects
- **Graceful degradation**: if embedding API is unavailable, falls back to FTS5-only search

## Embedding model migration ([6020])

If the daemon fails with `[6020] EmbeddingModelMismatch`, the database was last embedded with a different model than the one currently configured in `engram.toml`. The error message names both models and instructs the user to run `engram reembed`. As an agent, do not retry the same request — it will keep failing. Either:

- Ask the user to run `engram reembed` (one-shot CLI; recomputes every memory with the active provider, then records the new model in `schema_meta`).
- Or revert the config to the model named as `stored:` in the error message.

After `engram reembed` finishes cleanly, the next daemon start passes the guard and memory tools resume working.

## Local / Ollama providers (no API key)

If `engram.toml` sets `provider = "ollama"` for `[embedding]` and/or `[llm]`, Engram talks to a local Ollama daemon — there is no API key to configure or surface. Missing-key warnings do not apply to ollama. The host defaults to `http://localhost:11434` and is overridable via the `ENGRAM_OLLAMA_HOST` env var. If memory tools fail with a connection error under this provider, the Ollama daemon is likely not running or the model is not pulled (`ollama pull qwen3-embedding:0.6b`, `ollama pull qwen3:4b`) — ask the user to start it rather than retrying.