Expand description
MemFail-style per-operation fault-isolation harness.
§Anchor
MemFail frames a long-running agent’s recall pipeline as a chain of distinct operations — summarize → store → retrieve — and makes the per-operation behaviour the testable unit. A failure observed at the recall surface is decomposed into the single stage responsible for it.
mnemo’s operation seams are the three primitives exposed on
MnemoEngine:
- store =
MnemoEngine::remember— write a [MemoryRecord] into theStorageBackendand the vector + full-text indices, emit aMemoryWriteevent, link the hash chain. - summarize =
MnemoEngine::run_consolidation— cluster episodic records by tag overlap and replace each cluster with a structured[Consolidated from N memories] …semantic bundle (SourceType::Consolidation) plusconsolidated_fromrelations. Closest mnemo analogue to MemFail’s “summarize” stage. - retrieve =
MnemoEngine::recall— score the active bank under the active retrieval mode (hybrid RRF by default), return the top-k.
§What this harness is
A set of adversarial probes — one per operation — each engineered so a failed assertion is attributable to exactly one stage. The probes are run in order (store → summarize → retrieve); a downstream probe trusts its upstream peers because their probes already passed in the same run.
The canonical
run_stale_context_fixture case demonstrates the attribution
shape: write the same fact twice (older write at high importance,
newer write at low importance), then recall. The default hybrid
ranker returns the older / stale record on top. Store + summarize
probes succeed (both records are in storage with correct content
hashes; no consolidation has run), so the harness attributes the
stale recall to the retrieve stage — exactly the MemFail
“isolate the operation” output.
§What this harness is NOT
- Not a recall-quality benchmark. The probes target seams, not
retrieval-mode quality. Use
bench/locomofor quality numbers. - Not a faithful MemFail reproduction. The arXiv reference is prior-art-only; mnemo’s harness exercises the three primitives that the public MCP surface actually exposes.
- Not a write-side guard. A failed probe surfaces an attribution; gating production writes on it is the caller’s choice.
Structs§
- Attribution
Report - Output of
run_stale_context_fixture: the canonical MemFail case.attributed_stageis the single stage the harness blames for the observed failure based on which upstream probes still pass. - Probe
Outcome - Outcome of a single adversarial probe.
- Stage
Report - Result of running every probe in a single stage.
Enums§
- Stage
- Identifier for the three operation stages MemFail decomposes a recall into. The harness reports findings keyed on this enum so callers can pivot dashboards by stage.
Functions§
- run_
retrieve_ probes - Run the retrieve adversarial probe set against
engine. - run_
stale_ context_ fixture - Canonical MemFail attribution fixture.
- run_
store_ probes - Run the store adversarial probe set against
engine. - run_
summarize_ probes - Run the summarize adversarial probe set against
engine.