Skip to main content

Crate mnemo_engine

Crate mnemo_engine 

Source
Expand description

§Mnemo

Mnemo is an encrypted, single-file, portable agent-memory engine.

A whole memory store — vectors, content, metadata, and the multi-signal recall machinery an agent needs — lives in one file you can copy, back up, or hand to another process. The file is encrypted at rest with a two-tier key hierarchy: an Argon2id key-encryption key (KEK) derived from a passphrase wraps a random data-encryption key (DEK), and the DEK encrypts every page with AES-256-GCM.

§Quick start

use mnemo_engine::{Mnemo, MnemoConfig, Memory, MemoryType, RecallRequest};

let cfg = MnemoConfig { dimensions: 3, ..Default::default() };
let mut db = Mnemo::create("agent.mnemo", "correct horse battery", cfg)?;

db.remember(
    Memory::new("the user prefers dark mode", MemoryType::Semantic, vec![0.1, 0.2, 0.9])
        .with_agent("assistant-1")
        .with_importance(0.8),
)?;
db.flush()?;

let hits = db.recall(&RecallRequest::new(vec![0.1, 0.2, 0.9]).top_k(5))?;
for h in hits {
    println!("{:.3}  {}", h.score, h.memory.content);
}

§What is and is not built

This crate implements a real, tested core: the encrypted single-file storage engine, the crypto layer, the agent-memory model, multi-signal recall, an IVF+PQ approximate-nearest-neighbour index, a write-ahead log, snapshot-based point-in-time recovery, a bounded LRU page cache, and the Session conversation wrapper. Exact brute-force search remains available as the ground-truth baseline; a built index makes Mnemo::recall sub-linear. Each Mnemo::flush is one atomic, WAL-committed transaction and a restorable snapshot — Mnemo::restore_to rewinds the database to any past transaction. Python and TypeScript language bindings are the one documented roadmap item — see the README.

Modules§

essence_status
Status values the essence section supports (Phase 12.7b). More kinds (tasks: queued → active → ready → done | blocked; findings: open → accepted | deferred | rejected | needs-user → verified) ship with Phase 13 reusing the same status-record core; the transition tables for those are declared as constants in this crate but currently unused.
guide
GUIDE-1 (12.2): the guide for agents, compiled into the binary (docs/design/12.2-onboard.md, section 6).
keys
12.3: keys (docs/design/12.3-keys.md).
mcp
Hand-rolled Model Context Protocol (MCP) stdio server for MNemo.
redact
12.2 step 4: redacting known secret shapes from text before it’s stored (docs/design/12.2-onboard.md, section 5). The hooks’ transcript ingest always runs it; mnemo thread ingest --redact turns it on for a direct call.

Structs§

BuildInfo
Build-time cryptographic backend information.
CacheBudget
Per-namespace eviction budget. cache_put enforces both caps: on insert, the LRU is evicted until both live-entry count and live byte total fit under the caps.
CachePutOpts
Options accepted by Mnemo::cache_put. Defaults describe an untyped, never-expiring, non-cost-hinted entry.
CacheStats
Summary statistics for a namespace (or the whole cache).
CachedValue
A returned cache hit. accessed_at / access_count are updated on the directory before this is constructed, so the values here reflect the post-touch state (mirroring the recall-time convention).
CancelToken
Cancels a shared-mode call that is waiting for the file’s lock, from another thread: an iOS background activity’s expiry handler, say. The call releases every lock it holds and returns MnemoError::Cancelled.
CloseError
The error from Mnemo::try_close: the final flush failed, and the handle comes back with it, still open and still holding every write that wasn’t flushed.
CommitBatch
Atomic write batch for crate::Mnemo::commit_deltas (Phase 12.7c). The planner’s “one call that composes facts + essence + turns” shape: everything validates up front, then every write runs under a single flush().
CommitReport
Result of crate::Mnemo::commit_deltas (Phase 12.7c).
CompactReport
Result of a Mnemo::compact_file run.
EmbedCmd
Persistent-child-process adapter for a user-supplied embedder command. See the module docs for the protocol.
EmbedderIdentity
Describe an embedder’s observable identity. Used both in-memory on an open [Mnemo] handle and persisted at metadata.embedder on the scaffold manifest so that reopening with a different embedder is caught at set_embedder time instead of silently polluting the index.
EssenceOp
One essence op inside a CommitBatch (Phase 12.7c). Validation (missing/extra fields per kind) runs up front in crate::Mnemo::commit_deltas — a single bad op aborts the whole batch before any write lands.
ExportedTurn
One turn of a thread’s archive in the ingest shape (Phase 12.7d), as returned by crate::Mnemo::thread_export_turns and by Python db.thread_export, MCP thread_export and mnemo thread export.
Filter
Pre-scoring filter applied by crate::Mnemo::recall (Phase 12.6).
IndexConfig
Tuning knobs for building and querying an [IvfPqIndex].
IndexInfo
A read-only snapshot of an index’s shape, surfaced via Mnemo::stats.
IngestReport
Result of one crate::Mnemo::ingest_thread call (Phase 12.7a).
KdfParams
Argon2id cost parameters. Stored (unencrypted) in the file header so the KEK can be re-derived on open.
KeyedMemory
One keyed fact inside a CommitBatch (Phase 12.7c).
KeywordIndex
An in-memory keyword index over a set of texts, with the tokenizer and BM25 scoring (k1 1.2, b 0.75) of v9’s keyword recall, for tools that search what they’ve already read, without the file open (mnemo inspect). It needs no embedder and works on texts from any format. It uses tokenizer 2 (TOK-1), so a part of a dotted name matches.
Memory
A single memory entry.
MemoryPatch
Patch body for crate::Mnemo::update (Phase 12.1): every field is optional, and only the Some(...) fields overwrite the corresponding field on the target memory. All fields can be set independently.
Mnemo
An encrypted, single-file agent memory database.
MnemoConfig
Configuration for creating a new database.
RecallRequest
A retrieval request for Mnemo::recall.
RecallResponse
Phase 12.8 budgeted-recall envelope returned by Mnemo::recall_budgeted. Wraps the Vec<RecallResult> the legacy Mnemo::recall returns so callers can tell whether RecallRequest::max_bytes cut anything.
RecallResult
One scored result from Mnemo::recall.
ReembedReport
What Mnemo::reembed_file did (12.3 step 4, v0.15).
ScoreWeights
Weights for the multi-signal recall score (scoring v2, shipped in v0.7.0).
SemanticCachePutOpts
Options for crate::Mnemo::cache_put_semantic. Same as CachePutOpts plus a required model string — a hit from a different embedding model’s cache is a semantics bug, not a win, so the engine forces callers to declare the model and matches it on lookup.
Session
A scoped conversation session over a Mnemo database.
SharedMnemo
A brain opened in shared mode (12.4): see Mnemo::open_shared.
SharedOptions
Options for a shared-mode handle (12.4).
SnapshotInfo
A restorable point in a database’s history — see Mnemo::snapshots.
Stats
Summary statistics for a database.
StatsSummary
Mnemo::stats_summary: the cheap statistics, read from the catalog and the header (v0.12.2, RB-002.3).
ThreadTurn
One turn fed into crate::Mnemo::ingest_thread (Phase 12.7a).
TranscriptOptions
How read_transcript_with reads.
Turn
One conversation turn, written to working memory by Session::add_turn.
Ulid
Re-export of ulid::Ulid, the identifier type used for memories. A Ulid is a unique 128-bit lexicographically sortable identifier
VerifyReport
What Mnemo::verify_report checked.

Enums§

CacheFlushPolicy
Per-handle flush policy for cache writes. Memory writes ignore this and always behave as Strict.
EssenceOpKind
Which op a EssenceOp carries. Mirrors the per-method Mnemo::essence_* surface so one batch can mix section + timeline edits in one call.
EssenceScope
Scope selector for an essence item (Phase 12.7b). Thread-scoped items belong to one _mnemo.thread; project-scoped items are shared across every thread in the file.
EssenceSection
Which essence section an item belongs to.
FilterTerm
One AND-ed condition applied to a candidate memory’s metadata during filtered recall (Phase 12.6). Terms are combined conjunctively — a memory passes the filter only if every term evaluates true against its metadata map. See Filter for the surface API.
KeyScope
Scope selector for a keyed slot (Phase 12.1.1). A keyed remember’s slot is always identified by (scope_owner, key): Private(agent_id) slots are per-agent, Shared slots are per-file. history_in(key, scope) uses the same selector so the lookup matches the write exactly.
KeywordTokenizer
TOK-1: the state of a file’s keyword index, as info and verify show it.
MemoryType
The four kinds of agent memory.
Metric
Distance/similarity metric for vector comparison.
MnemoError
All errors the Mnemo engine can produce.
RecallMode
MOBILE-2d: how recall scores vectors, from Mnemo::stats.
Role
Who produced a conversation turn.
Scope
Visibility of a memory across agents sharing one .mnemo file.
SecretKind
12.3: what kind of secret a caller handed the engine. A passphrase is a person’s; a key is a uniformly random 256-bit value a machine made.
ToolResults
What a Claude Code transcript’s tool results keep (12.2 step 4).
TranscriptFormat
The supported transcript shapes for read_transcript / mnemo thread ingest --format ….
TurnPolicy
What crate::Mnemo::commit_deltas does with the turns carried in the batch (Phase 12.7c).
WriteOutcome
Outcome of a 12.1 write operation (remember with key, or update).

Constants§

CONTINUATION_TIMELINE_ENTRIES
How many timeline entries from the predecessor thread the continuation render pulls in (Phase 12.7c). Small enough to keep the render under the default 6 KB budget, large enough to carry a handful of milestones.
DEFAULT_BATCH_MAX_AGE
Default max_age for CacheFlushPolicy::Batched.
DEFAULT_BATCH_MAX_DIRTY
Default max_dirty for CacheFlushPolicy::Batched.
DEFAULT_EMBED_TIMEOUT
Default per-request timeout. Set generously because a cold-start sentence-transformers call can genuinely take a few seconds the first time; configurable via EmbedCmd::with_timeout (Rust API) or the --embed-timeout CLI flag.
DEFAULT_KEYWORD_WEIGHT
FMT-9d: default weight ε of the normalised BM25 score in hybrid recall (relevance = similarity + ε · bm25 / max bm25).
DEFAULT_MAX_BYTES
Default per-namespace byte budget (values + directory overhead). Once exceeded, cache_put evicts LRU until the total falls back under the cap.
DEFAULT_MAX_ENTRIES
Default per-namespace entry-count budget. Once a namespace holds this many live entries, cache_put evicts the LRU entry before inserting.
DEFAULT_SEMANTIC_THRESHOLD
Default similarity threshold for crate::Mnemo::cache_get_semantic. Lookups below this are treated as misses. Chosen conservatively to avoid false hits on subtly-different prompts; callers can override per-lookup. Model-dependent — text-embedding-ada-002 and BGE at this threshold rarely surface unrelated content, but if you’re working with a lossier embedder consider raising it.
DEFAULT_TEXT_KEYWORD_WEIGHT
12.3 step 4 (v0.15): default weight of the BM25 score when it comes from a recall’s own text (hybrid by default, no keywords given). Lower than DEFAULT_KEYWORD_WEIGHT because a question’s common words (“how”, “the”) match unrelated memories: on this project’s brain at 0.5 a memory with similarity 0.18 came second on “how releases are approved”; at 0.25 the right fact led all three DOGFOOD-0 questions with no such intruder (docs/design/12.3-retrieval.md, section 5).
DEFAULT_TURN_COMPRESS_AFTER_SECS
FMT-9e: by default a thread turn is written plain and compressed once it’s a week old (by its created_at), so a live session’s recent turns read at full speed and archived history is small.
DEFAULT_VECTOR_CACHE_BUDGET
FMT-9b: default Mnemo::set_vector_cache_budget: the fp16 vector column stays resident while it fits (100K × 384 dims is about 77 MB) and is read through the page cache above it. PLANNER-A, 2026-11-07.
ESSENCE_DEFAULT_RENDER_BYTES
Default byte budget for crate::Mnemo::essence rendering (Phase 12.7b). Matches the planner’s “max_bytes = 6144” spec, which sits comfortably below a 4K-token context window.
ESSENCE_SECTION_MAX_BYTES
Maximum content length for a section item (Phase 12.7b). intent / conventions / decisions / state / open / next / lessons all share the same single-line cap. Items over this are rejected.
ESSENCE_TIMELINE_MAX_BYTES
Maximum content length for a timeline entry (Phase 12.7b). Entries over this are rejected, not truncated — the planner’s spec is explicit about this because truncation would lose the narrative, and splitting across records would lose ordering.
ESSENCE_TRANSITIONS
Essence transition table for Phase 12.7b. Only open → resolved is allowed. Timeline entries don’t use this (they’re append-only and have no status).
FINDING_TRANSITIONS
Finding transition table (declared for Phase 13; currently unused in the engine). open → {accepted | deferred | rejected | needs-user} → verified.
INDEX_PERSIST_EVERY_CHANGES
PERF-1: …or once this many vectors were added, changed or removed since the last persist. Records written after the persisted index are re-assigned when the index is next loaded, so this bounds that work.
INDEX_PERSIST_EVERY_COMMITS
PERF-1: the ANN index is derived data, so flush persists it only every this many commits that changed it (or see INDEX_PERSIST_EVERY_CHANGES, Mnemo::flush_index, Mnemo::close). Between persists, only the catalog is rewritten.
MNEMO_META_CONTINUES
Reserved _mnemo.continues key (Phase 12.7c). Set on a thread-start marker record to point at the previous thread whose context the new thread continues from. Value: the prior thread id (string). Rendering a continued thread pulls the prior thread’s open items, next and last-N timeline entries ahead of the new thread’s own essence.
MNEMO_META_INVALID_AT
Set on the earlier record when superseded. Value: unix seconds (i64). Not interpreted by the engine’s scorer — purely for audit / history.
MNEMO_META_KEY
Upsert slot name on a remember(…, key = K) write (12.1). Combined with the memory’s scope (and agent_id for Private scope) to identify a single live memory per slot: a newer remember(key=K) supersedes the prior one.
MNEMO_META_KIND
Record kind for thread-archive turns (Phase 12.7a). Value: "turn".
MNEMO_META_KIND_ESSENCE
Marker value for _mnemo.kind on essence records (Phase 12.7b).
MNEMO_META_KIND_THREAD_LINK
Marker value for _mnemo.kind on a thread-start record (one per thread_start(new, --from old) call, carrying the link in _mnemo.continues).
MNEMO_META_KIND_TURN
Marker value for _mnemo.kind on thread-turn records.
MNEMO_META_LENGTH
Byte length of the original tool-result content (Phase 12.7a). Set alongside _mnemo.sha256 when truncation happened; absent otherwise.
MNEMO_META_NS
Namespace prefix used for every v0.8.0+ reserved metadata key (R-001.10). Keys outside this namespace are untouched user metadata.
MNEMO_META_PART
Zero-based part number for a turn that was chunked at the 8 KB boundary during ingest (Phase 12.7a). Absent → the turn fits in a single record.
MNEMO_META_REFS
Reserved _mnemo.refs key — optional array of free-form reference strings (turn ids, memory ids, commits, paths) attached to an essence record (Phase 12.7b).
MNEMO_META_ROLE
Role of the turn’s author (Phase 12.7a). Value: "user", "assistant", "system", "tool", or any other string the source transcript used.
MNEMO_META_SECTION
Reserved _mnemo.section key — which essence section an item belongs to (Phase 12.7b). Values: "timeline", "intent", "conventions", "decisions", "state", "open", "next", "lessons". Timeline entries are append-only.
MNEMO_META_SEQ
Monotonically increasing sequence number within the thread (Phase 12.7a). Value: unsigned integer, 0-based. Lets thread export reconstruct the transcript in order.
MNEMO_META_SHA256
Hex-encoded SHA-256 of the original tool-result content (Phase 12.7a). Set when the content exceeded the 16 KB truncation cap during ingest; absent otherwise.
MNEMO_META_STATUS
Reserved _mnemo.status key — current status of a status-record. Values are per-kind (Phase 12.7b: essence uses "open" and "resolved"). Timeline entries never carry this.
MNEMO_META_STATUS_HISTORY
Reserved _mnemo.status_history key — array of status transitions, each {"status": <str>, "at": <unix_secs>}. Append-only — the first entry records creation, each later entry records a transition.
MNEMO_META_SUPERSEDED_BY
Set on the earlier record when a newer remember(key=K) has replaced it. Value: the new record’s ULID string. Together with _mnemo.invalid_at this marks the record as “superseded, not current” — default recall, about and the key index skip it; history(key) walks the chain.
MNEMO_META_SUPERSEDES
Set on the newly-inserted record when a remember(key=K) supersedes an earlier record. Value: the superseded record’s ULID string.
MNEMO_META_THREAD
Thread identifier the record belongs to (Phase 12.7a). Value: a caller-supplied string, typically the Claude Code session uuid or the planner-chosen thread slug.
MNEMO_META_TURN_ID
Opaque per-turn identifier from the source transcript (Phase 12.7a). Value: typically a uuid4. Used for idempotence — ingest_thread skips any turn whose turn_id already exists on a record in the target thread.
MNEMO_META_UPDATED_AT
Set on a record each time Mnemo::update succeeds. Value: unix seconds. Lets callers distinguish an unchanged-since-creation record from one that has been edited in place. Not interpreted by the scorer.
TASK_TRANSITIONS
Task transition table (declared for Phase 13; currently unused in the engine). queued → active → ready → done | blocked, with blocked → active as the only back-edge.
THREAD_TOOL_RESULT_TRUNCATE_BYTES
Maximum content size for a tool-result turn before truncation (Phase 12.7a). Tool results frequently run tens of kilobytes; stored truncated with _mnemo.sha256 + _mnemo.length so the original can be recovered (or verified) out-of-band.
THREAD_TURN_CHUNK_BYTES
Maximum content size for a single turn record before chunking kicks in (Phase 12.7a). 8 KB per chunk keeps each record small enough that the per-record page cost dominates — and so that a thread of mostly short messages ingests as one record each.
V9_ANN_MIN_FACTS
FMT-9b: below this many facts a v9 file has no ANN index: an exact fp16 scan of 20K × 384 dims is about a millisecond, and the IVF+PQ codebook alone would cost more disk than it saves time.
V9_RERANK_PER_HIT
FMT-9f (PLANNER-A, 2026-11-08): on v9 an unpinned recall reranks at least this many candidates per requested hit (500 for top_k 10), or the index’s own auto depth when that’s deeper (above 100K). The v9 index is trained on fp16-rounded vectors, and at 20K–50K the auto depth (×10–×25) lost up to 0.03 recall@10 against v8; ×50 is within 0.005 at every measured tier. v8 is unchanged.

Traits§

Embedder
Minimal trait any embedder implementation must satisfy. Returns vectors in the same order as the input texts.

Functions§

build_info
Report this build’s cryptographic backend configuration.
read_transcript
Read a transcript from any Read in the given format and return the turns in file order. Non-matching / unrecognized line types are skipped silently (Claude Code adds new type values over time).
read_transcript_with
read_transcript with TranscriptOptions.

Type Aliases§

Result
Convenience result alias used throughout the crate.