# System Architecture
## Summary
The target system should separate trusted knowledge, raw memory capture, distilled memory, retrieval, and policy enforcement into distinct layers.
## Architectural Layers
### 1. Knowledge Layer
Primary store: Obsidian vault
Purpose:
- trusted long-term notes
- manually curated project knowledge
- persistent developer and team preferences
- ADRs, runbooks, and postmortems
### 2. Memory Ledger Layer
Primary store: local append-only event store
Purpose:
- session transcripts
- command traces
- commit metadata
- decision candidates
- captured incidents and debugging trails
This layer should not write directly into curated Obsidian notes.
### 3. Distillation Layer
Purpose:
- convert raw events into memory objects
- derive wake-up packets
- produce room-level summaries
- detect stale or conflicting knowledge
Near-term boundary:
- wake-up packet v1 should be derivable from curated vault notes even before the ledger exists
- later ledger signals may enrich packets, but should not replace curated source-of-truth notes by default
### 4. Retrieval Layer
Purpose:
- query by task, repo, files, and memory type
- combine structural signals with semantic or lexical signals
- produce ranked candidates
- return evidence with provenance
### 5. Gateway Layer
Interfaces:
- CLI
- MCP server
- local HTTP or Unix socket API
- export format for client-specific integrations
### 6. Policy and Security Layer
Purpose:
- classify sensitivity
- redact secrets
- filter outputs by client and mode
- log what was returned and why
## Current Repository Mapping
### Already Present
- CLI request handling
- project routing from `cwd`
- vault scanning
- note scoring and selection
- multiple render formats
- experimental GUI
### Missing
- memory ledger
- distillation jobs
- client policy engine
- MCP server
- cache invalidation policy
- semantic retrieval path
## Retrieval Strategy
The retrieval engine should gradually evolve into a hybrid stack:
1. hard filters
- project
- note roots
- sensitivity policy
- memory type
2. structural scoring
- title
- headings
- wikilinks
- tags
- related files
- source-of-truth flags
3. optional semantic ranking
- local embeddings
- approximate nearest neighbor index
4. evidence packaging
- wake-up packet
- ranked note set
- raw evidence links
## Cache Strategy
The current in-process scan cache is acceptable as an intermediate step, but the long-term design should include:
- vault fingerprinting
- invalidation by file modification time
- optional persistent local index
- separation between scan cache and retrieval index cache
## Security Model
### Data Handling Tiers
- `public`
- `internal`
- `confidential`
- `secret`
### Enforcement Points
- before capture
- before indexing
- before retrieval
- before render/export
### Required Controls
- local-only mode
- secret redaction
- provenance preservation
- retrieval audit log
## Gateway Design Direction
The recommended next stable interface is MCP.
Suggested tool surface:
- `memory_search`
- `memory_get`
- `memory_wakeup`
- `memory_explain`
- `memory_record`
This keeps `spool` model-agnostic while still usable from AI clients that support tool calling.
## Deployment Modes
### Mode 1: CLI Only
Best for local scripts and direct terminal usage.
### Mode 2: MCP Server
Best for Claude, Codex, and compatible tool-based clients.
### Mode 3: Local Daemon
Best when caching, indexing, and wake-up generation should persist across sessions.
## Recommended Near-Term Architecture
1. keep CLI stable
2. define and implement wake-up packet generation on top of the current retrieval core
3. add MCP interface on top of the same core and packet contract
4. design a local ledger store beside Obsidian
5. add cache invalidation and a persistent index
This order is intentional:
- wake-up packets define the first stable structured memory contract
- MCP should expose that contract rather than inventing a separate one first
- the ledger can evolve later without redefining the external packet shape