rusty_agent_runtime/memory.rs
1//! Governed memory (R0.8 Rusty Learn, waves 1–2): the record model, the
2//! structured retrieval contract with its token-bounded deterministic
3//! assembly, the journaled read/write seam, and — from wave 2 — the
4//! correction loop and the memory operations (consolidation, conflict
5//! detection, forgetting) as journaled, evidence-carrying transitions.
6//!
7//! The design doc is `docs/learn-design.md` ("Governed memory"); the
8//! learning rule it answers to: **no learning process may silently rewrite a
9//! production prompt, graph, policy, memory, or tool permission.** Memory is
10//! therefore a governed record store, not a JSON blob an agent edits through
11//! a tool call:
12//!
13//! - [`MemoryRecord`] — one serde-versioned, content-addressed record:
14//! identity is `sha256` over content plus provenance, so a changed record
15//! is a new id and there is no in-place update anywhere in the model.
16//! Supersession is a chain of immutable records; the superseded record is
17//! retained as evidence but filtered from default retrieval.
18//! - [`MemoryScope`] — the five-scope taxonomy: `run` / `agent` / `team` /
19//! `user` / `tenant`. It maps onto R0.7's four [`crate::agents::StateScope`]s
20//! with one honest adaptation, stated exactly as the design draws it:
21//! `StateScope` has no `run` member, because state outlives runs by
22//! design; run scope is new here (memory whose lifetime is bound to one
23//! run's thread); and **`agent` scope is `StateScope::Private` under its
24//! memory name**. The taxonomy is a superset, not a fork: an agent
25//! manifest's declared `StateScope`s translate one-to-one into the memory
26//! scopes it may write ([`MemoryScope::from_state_scope`]).
27//! - [`MemoryProvenance`] — who wrote it (`agent:{id}` / `human:{id}` /
28//! `distiller:{name}` / `system`), from what evidence (run id + journal
29//! event ids, a correction id, a candidate id, the source record ids of a
30//! consolidation), and when. Mandatory: a record that cannot name its
31//! origin cannot be audited.
32//! - [`MemoryQuery`] + [`ContextBudget`] — deliberately structural
33//! retrieval (scope, kind, key/tag equality, validity-at-time, minimum
34//! confidence, exclude-expired, exclude-superseded, authored-by; **no
35//! similarity search** in R0.8), packed into a prompt assembly by a
36//! deterministic rank (explicit priority, then confidence, then recency,
37//! with the content address as the final tie-break) until the token budget
38//! is exhausted.
39//!
40//! # The journaled seam
41//!
42//! A memory read is an [`Effect::ReadOnly`] effect journaled as
43//! [`RunEventKind::MemoryRead`], with the resolved query plus budget as the
44//! input payload and the assembly itself — the record ids and their order,
45//! plus the token accounting — as the output payload. Two properties the
46//! design requires follow from this module's discipline:
47//!
48//! - **Determinism.** Equal store state and equal budget produce byte-equal
49//! assemblies ([`assemble`] is a pure function with a total order), and
50//! the resolved query — including the `as_of` timestamp, stamped through
51//! the run's clock when unset — travels in the journaled request, so the
52//! request hash fully determines the result.
53//! - **Replay-serving.** Exact replay serves the journaled assembly instead
54//! of re-querying the store, per the rule the Flight Recorder already
55//! applies to journaled model and tool calls. [`MemoryReplaySource`] is
56//! the serving cursor (sequence + request hash, the same matching rule
57//! [`crate::replay::ReplaySource`] applies to model/tool calls), and
58//! [`JournaledMemory::read`] re-journals the served event byte-identically
59//! — including clock-read parity with the live path, so a logical clock's
60//! tick sequence stays aligned with the recorded run.
61//!
62//! A memory write is an [`Effect::Idempotent`] effect under the derived key
63//! `memory:{scope}:{memory_id}` ([`memory_effect_key`]): retried submissions
64//! converge, and the write is journaled as [`RunEventKind::MemoryWrite`]
65//! with its provenance. Writes are never served during replay — a replayed
66//! run issuing a write has diverged from its evidence.
67//!
68//! # The correction loop and the memory operations (wave 2)
69//!
70//! Wave 2 lands the correction loop's record-plane half: [`Correction`] is
71//! the highest-trust input the learning system has, so the loop's rule is
72//! **a correction becomes an attributed candidate memory or example — never
73//! an in-place rewrite of what it corrects**. The derived record carries
74//! `human:{author}` provenance with the correction id in evidence (the
75//! design's `human:{author} via correction:{id}` attribution), confidence
76//! 1.0, and — at agent scope or wider — a [`Candidacy`] mark: candidacy,
77//! not adoption, because a wrong human correction at tenant scope is a
78//! production incident with a name attached and evaluation is cheap
79//! insurance. Run scope adopts directly: it affects only the run that
80//! produced it. Same-key correction-sourced writes auto-supersede the prior
81//! record (open question 5's asymmetry: corrections are trusted because
82//! they are attributed; distillations are not because they are inferred).
83//!
84//! The three memory operations are journaled transitions over the store,
85//! never background daemons:
86//!
87//! - **Consolidation** — [`consolidation_summary`] builds the one `summary`
88//! record that distills N sources: it names them in
89//! [`MemoryEvidence::source_memory_ids`] (which is also what makes
90//! dependent-summary invalidation computable on forgetting) and
91//! supersedes them — [`apply_query`] treats a summary's sources as
92//! superseded, so default retrieval serves the summary alone. The
93//! distillation semantics stay with the caller; the runtime owns the
94//! record's invariants.
95//! - **Conflict detection** — [`detect_conflicts`] flags live records that
96//! share a key, overlap in validity, and carry contradictory content.
97//! It flags ([`MemoryConflict`], a review item); it never resolves.
98//! - **Forgetting** — [`plan_forget`] computes the erasure before anything
99//! is deleted: the targets plus the dependent summaries invalidated by
100//! walking the source naming in reverse, transitively. Deletion is real
101//! (derived state is erasable; run journals are hash-chained evidence and
102//! are not — open question 4), and the receipt is the journaled
103//! [`RunEventKind::MemoryForget`] tombstone: [`MemoryForgetTombstone`]
104//! carries the id, scope, reason, and dependent invalidations — metadata
105//! by construction, with no content field a careless serializer could
106//! leak the forgotten bytes through.
107//!
108//! # Token accounting
109//!
110//! The budget is enforced in **estimated tokens: serialized content bytes ÷
111//! 4, plus a declared safety margin** ([`ContextBudget::margin_percent`],
112//! default [`DEFAULT_TOKEN_MARGIN_PERCENT`]), recorded as such on every
113//! assembly ([`TokenAccounting`]). This is the design's open-question-3
114//! default, stated plainly: provider-precise tokenizers differ per model and
115//! are heavyweight; model-precise counting plugs in later behind the same
116//! [`ContextBudget`] type, and until then the margin is the honest hedge.
117//!
118//! Golden-file tests under `tests/golden/` pin every wire shape in this
119//! module; any accidental contract drift fails CI.
120
121use std::collections::{BTreeMap, HashSet};
122use std::sync::{Arc, Mutex, MutexGuard};
123
124use async_trait::async_trait;
125use chrono::{DateTime, Utc};
126use serde::{Deserialize, Serialize};
127use serde_json::{json, Value};
128
129use crate::agents::StateScope;
130use crate::error::{Result, RustyError};
131use crate::journal::{EventDraft, Journal, JournalSnapshot};
132use crate::record::{
133 sha256_hex, ArtifactRef, Effect, PayloadRef, RunEvent, RunEventKind, INLINE_PAYLOAD_MAX_BYTES,
134};
135use crate::replay::ServedEffect;
136
137/// The memory schema version a run pins in its manifest
138/// ([`crate::record::RunManifest::memory_schema`]). Bump only on a breaking
139/// change to the record model; additive evolution uses serde defaults so
140/// previously written records keep deserializing.
141pub const MEMORY_SCHEMA_VERSION: &str = "memory-v1";
142
143/// The divisor of the estimated-token accounting: estimated tokens are
144/// serialized content bytes ÷ `TOKEN_BYTES_PER_ESTIMATE`, rounded up, before
145/// the safety margin. See the module docs for why the estimate — not a
146/// provider-precise tokenizer — is the R0.8 default.
147pub const TOKEN_BYTES_PER_ESTIMATE: u64 = 4;
148
149/// The default safety margin of the estimated-token accounting, in percent
150/// (design open question 3's "declared safety margin"): a record's estimated
151/// cost is `ceil(bytes / 4) * (1 + margin)`, so tokenizer drift against the
152/// estimate eats the margin before it eats the budget.
153pub const DEFAULT_TOKEN_MARGIN_PERCENT: u32 = 20;
154
155fn invalid(message: impl Into<String>) -> RustyError {
156 // A memory write is a state update to the governed store; contract
157 // validation failures reuse the invalid-update class rather than
158 // growing the error taxonomy for one module.
159 RustyError::InvalidUpdate(message.into())
160}
161
162fn replay_error(message: impl Into<String>) -> RustyError {
163 RustyError::Replay(message.into())
164}
165
166// --------------------------------------------------------------------- //
167// The record model
168// --------------------------------------------------------------------- //
169
170/// What a memory record is. Closed enum — retrieval and consolidation match
171/// exhaustively on it.
172#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
173#[serde(rename_all = "snake_case")]
174pub enum MemoryKind {
175 /// An asserted fact about the world the agent operates in.
176 Fact,
177 /// A stated preference (a user's, a team's), meant to shape future
178 /// behavior within its scope.
179 Preference,
180 /// A corrected input/output pair — the correction loop's output (R0.8
181 /// wave 2): the input a run saw and the behavior it should have
182 /// produced.
183 Example,
184 /// Consolidation's output: one record distilling N sources, which it
185 /// names in [`MemoryEvidence::source_memory_ids`] — the naming is what
186 /// makes dependent-summary invalidation computable on forgetting.
187 Summary,
188}
189
190/// The governance state of a correction-derived record (R0.8 wave 2).
191/// Closed enum with the single honest state this wave needs — wave 3's
192/// evaluation outcomes extend it additively, the same evolution rule
193/// [`RunEventKind`] follows. Not part of the content address: candidacy is
194/// governance state, not identity — the same reading `tags` and `priority`
195/// already take.
196#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
197#[serde(rename_all = "snake_case")]
198pub enum Candidacy {
199 /// Attributed and stored, but unevaluated. The design's scope-decides-
200 /// the-path rule: a correction at agent scope or wider becomes a
201 /// candidate, because a wrong human correction at tenant scope is a
202 /// production incident with a name attached, and the evaluation step is
203 /// cheap insurance against it.
204 Pending,
205}
206
207/// The scope a memory lives at: whose memory it is and how far its
208/// visibility reaches. Closed enum; the superset mapping onto R0.7's
209/// [`StateScope`] taxonomy is the module's load-bearing contract (see the
210/// module docs and [`MemoryScope::from_state_scope`]).
211#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
212#[serde(rename_all = "snake_case")]
213pub enum MemoryScope {
214 /// Memory whose lifetime is bound to one run's thread. New in R0.8:
215 /// `StateScope` has no `run` member, because state outlives runs by
216 /// design. Run scope is written by the runtime on the run's behalf.
217 Run,
218 /// One agent's own memory — `StateScope::Private` under its memory
219 /// name. Scope authorization is the manifest check: an agent may write
220 /// agent scope only when its `CapabilityManifest` declares
221 /// `StateScope::Private`.
222 Agent,
223 /// Memory shared by a team's members — `StateScope::Team` under its
224 /// memory name, under the same turn discipline that governs team state.
225 Team,
226 /// One end user's memory, shared across that user's agents and threads —
227 /// `StateScope::User` under its memory name.
228 User,
229 /// Configuration-grade memory for the whole tenant — `StateScope::Tenant`
230 /// under its memory name. Writable by operators, not by agents; tenant
231 /// isolation is the `{tenant}/` id-namespacing unchanged.
232 Tenant,
233}
234
235impl MemoryScope {
236 /// The memory scope a declared [`StateScope`] translates into, one-to-one
237 /// (`private` → `agent`): the manifest check's mapping. `None` is
238 /// unreachable — the four `StateScope`s all have memory names — but the
239 /// signature keeps the mapping honest about direction: this is the
240 /// superset reading the subset.
241 pub fn from_state_scope(scope: StateScope) -> Option<MemoryScope> {
242 Some(match scope {
243 StateScope::Private => MemoryScope::Agent,
244 StateScope::Team => MemoryScope::Team,
245 StateScope::User => MemoryScope::User,
246 StateScope::Tenant => MemoryScope::Tenant,
247 })
248 }
249
250 /// The [`StateScope`] this memory scope corresponds to, when it has one:
251 /// `agent` → `private`, `team` → `team`, `user` → `user`,
252 /// `tenant` → `tenant`, and `run` → `None` — run scope has no
253 /// `StateScope` member because state outlives runs by design.
254 pub fn to_state_scope(self) -> Option<StateScope> {
255 match self {
256 MemoryScope::Run => None,
257 MemoryScope::Agent => Some(StateScope::Private),
258 MemoryScope::Team => Some(StateScope::Team),
259 MemoryScope::User => Some(StateScope::User),
260 MemoryScope::Tenant => Some(StateScope::Tenant),
261 }
262 }
263}
264
265/// A [`MemoryScope`] plus the concrete scope id (agent id, user id, team id,
266/// run id, or the tenant name). Scope ids are tenant-relative names: the
267/// store namespaces every record under its tenant, so an id never needs to
268/// re-encode the tenant it belongs to.
269#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
270pub struct ScopeAddress {
271 /// The taxonomy member.
272 pub scope: MemoryScope,
273
274 /// The concrete id inside the scope (e.g. `researcher-7` for
275 /// `agent`-scoped memory).
276 pub id: String,
277}
278
279impl ScopeAddress {
280 /// Build an address; validation of the id happens at the write gate,
281 /// not here (addresses also arrive from stored evidence).
282 pub fn new(scope: MemoryScope, id: impl Into<String>) -> Self {
283 Self {
284 scope,
285 id: id.into(),
286 }
287 }
288
289 /// The canonical `"{scope}:{id}"` string form, used in derived effect
290 /// keys (`memory:{scope}:{memory_id}`) and human-facing provenance.
291 pub fn as_address(&self) -> String {
292 let scope = serde_json::to_value(self.scope)
293 .ok()
294 .and_then(|v| v.as_str().map(str::to_owned))
295 .unwrap_or_else(|| format!("{:?}", self.scope).to_lowercase());
296 format!("{scope}:{}", self.id)
297 }
298}
299
300/// Who wrote a memory record. Closed enum: a record that cannot name its
301/// origin cannot be audited, so provenance is mandatory, not optional — and
302/// the attribution travels with the record through every later consumer
303/// (distiller, evaluator, auditor).
304#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
305#[serde(tag = "type", rename_all = "snake_case")]
306pub enum ProvenanceAuthor {
307 /// An agent wrote it (in a governed write inside a run): `agent:{id}`.
308 Agent {
309 /// The writing agent's id (tenant-relative).
310 agent_id: String,
311 },
312 /// A human wrote it — a correction, the highest-trust input the learning
313 /// system has: `human:{id}`. Human-authored records default to
314 /// confidence 1.0 (the claim is the person's, stated plainly).
315 Human {
316 /// The correcting human's identity. Mandatory because a correction
317 /// that cannot name its corrector is indistinguishable from a
318 /// prompt edit.
319 human_id: String,
320 },
321 /// A distiller produced it from recorded evidence: `distiller:{name}`.
322 Distiller {
323 /// The distiller's name (application code, not runtime code).
324 name: String,
325 },
326 /// The runtime itself wrote it (e.g. a run-scope record written on the
327 /// run's behalf): `system`.
328 System,
329}
330
331impl ProvenanceAuthor {
332 /// The canonical id string (`agent:{id}` / `human:{id}` /
333 /// `distiller:{name}` / `system`), as the design's provenance
334 /// vocabulary spells it.
335 pub fn as_id_string(&self) -> String {
336 match self {
337 ProvenanceAuthor::Agent { agent_id } => format!("agent:{agent_id}"),
338 ProvenanceAuthor::Human { human_id } => format!("human:{human_id}"),
339 ProvenanceAuthor::Distiller { name } => format!("distiller:{name}"),
340 ProvenanceAuthor::System => "system".to_owned(),
341 }
342 }
343}
344
345/// What a memory record was derived from: the evidence an auditor walks.
346/// Every field is optional and absent from the wire when unset — a
347/// human-authored fact may carry no derivation at all, and sparse evidence
348/// must not change the shape for dense readers.
349#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
350pub struct MemoryEvidence {
351 /// The run whose journal the record was distilled or corrected from.
352 #[serde(default, skip_serializing_if = "Option::is_none")]
353 pub run_id: Option<String>,
354
355 /// The journal event ids inside `run_id` the record draws on.
356 #[serde(default, skip_serializing_if = "Vec::is_empty")]
357 pub event_ids: Vec<String>,
358
359 /// The correction (R0.8 wave 2) the record was derived from.
360 #[serde(default, skip_serializing_if = "Option::is_none")]
361 pub correction_id: Option<String>,
362
363 /// The candidate (R0.8 wave 3) whose promotion carried this record.
364 #[serde(default, skip_serializing_if = "Option::is_none")]
365 pub candidate_id: Option<String>,
366
367 /// The records a `summary` consolidated — the naming that makes
368 /// dependent-summary invalidation computable on forgetting.
369 #[serde(default, skip_serializing_if = "Vec::is_empty")]
370 pub source_memory_ids: Vec<String>,
371}
372
373impl MemoryEvidence {
374 /// `true` when no evidence is carried (the sparse wire shape).
375 pub fn is_empty(&self) -> bool {
376 self.run_id.is_none()
377 && self.event_ids.is_empty()
378 && self.correction_id.is_none()
379 && self.candidate_id.is_none()
380 && self.source_memory_ids.is_empty()
381 }
382}
383
384/// A record's origin: who wrote it, from what evidence, and when. Part of
385/// the content address ([`MemoryRecord::new`]) — two records with identical
386/// content but different origins are different records, because they answer
387/// differently to an audit.
388#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
389pub struct MemoryProvenance {
390 /// Who wrote it.
391 pub author: ProvenanceAuthor,
392
393 /// What it was derived from. Absent from the wire when empty.
394 #[serde(default, skip_serializing_if = "MemoryEvidence::is_empty")]
395 pub evidence: MemoryEvidence,
396
397 /// When the system learned it. Distinct from
398 /// [`ValidityWindow::valid_from`] (when the record claims to be true) —
399 /// the bitemporal split, kept as two plain timestamps (Zep's validity
400 /// window as a flat field).
401 pub written_at: DateTime<Utc>,
402}
403
404/// The interval the record claims to be true: `[valid_from, valid_until)`,
405/// open-ended when `valid_until` is `None`. Contradiction is handled by
406/// time (and supersession), never by deletion.
407#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
408pub struct ValidityWindow {
409 /// Inclusive start of the claimed-true interval.
410 pub valid_from: DateTime<Utc>,
411
412 /// Exclusive end of the claimed-true interval; `None` = open-ended.
413 #[serde(default, skip_serializing_if = "Option::is_none")]
414 pub valid_until: Option<DateTime<Utc>>,
415}
416
417impl ValidityWindow {
418 /// A window starting at `valid_from`, open-ended.
419 pub fn starting(valid_from: DateTime<Utc>) -> Self {
420 Self {
421 valid_from,
422 valid_until: None,
423 }
424 }
425
426 /// Whether the window claims to be true at `t`: inclusive start,
427 /// exclusive end.
428 pub fn contains(&self, t: DateTime<Utc>) -> bool {
429 self.valid_from <= t && self.valid_until.is_none_or(|until| t < until)
430 }
431}
432
433fn is_false(value: &bool) -> bool {
434 !*value
435}
436
437fn is_zero(value: &i64) -> bool {
438 *value == 0
439}
440
441/// One governed memory record: content-addressed, immutable, scoped,
442/// attributed.
443///
444/// Every field exists because a downstream operation needs it (the design's
445/// rule): retrieval filters on scope/kind/key/tags/confidence/validity/
446/// expiration/supersession/author; the assembly ranks on
447/// priority/confidence/recency; the audit walks provenance; forgetting walks
448/// `supersedes` in reverse. The one reserved field is `embedding` —
449/// reserved so vector retrieval slots in additively when the roadmap's
450/// de-prioritization lifts; nothing in R0.8 reads it.
451#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
452pub struct MemoryRecord {
453 /// The content address: `sha256` over the canonical serialization of the
454 /// record's content plus provenance (the one hashing primitive shared
455 /// with artifact references and journal heads). Immutable by
456 /// construction: a changed record is a new id.
457 pub memory_id: String,
458
459 /// What the record is.
460 pub kind: MemoryKind,
461
462 /// Whose memory it is.
463 pub scope: ScopeAddress,
464
465 /// The writer-declared lookup key, when the record answers a named
466 /// question ("what is user-7's timezone"). With no vector retrieval in
467 /// R0.8, keying is deliberate: absence of a hit is absence of a key, not
468 /// absence of a fact.
469 #[serde(default, skip_serializing_if = "Option::is_none")]
470 pub key: Option<String>,
471
472 /// Writer-declared tags; retrieval matches by equality (a record must
473 /// carry every queried tag).
474 #[serde(default, skip_serializing_if = "Vec::is_empty")]
475 pub tags: Vec<String>,
476
477 /// The writer-declared explicit priority — the assembly rank's first
478 /// input (higher packs first). `0` is the default and stays absent from
479 /// the wire, so pre-priority records keep their shape.
480 #[serde(default, skip_serializing_if = "is_zero")]
481 pub priority: i64,
482
483 /// Who wrote it, from what evidence, and when. Mandatory.
484 pub provenance: MemoryProvenance,
485
486 /// The writer-declared confidence in `(0, 1]`. Retrieval filters on it;
487 /// nothing in the runtime *computes* it — confidence is a claim, not a
488 /// measurement, and the model is honest about that.
489 pub confidence: f64,
490
491 /// The interval the record claims to be true.
492 pub validity: ValidityWindow,
493
494 /// When the system learned it (the bitemporal split's other timestamp;
495 /// duplicates [`MemoryProvenance::written_at`] deliberately — the record
496 /// is self-contained when provenance is summarized away by a consumer).
497 pub created_at: DateTime<Utc>,
498
499 /// Optional TTL. Expiration is a retrieval filter (and, from wave 2, a
500 /// forgetting trigger), not a silent reaper.
501 #[serde(default, skip_serializing_if = "Option::is_none")]
502 pub expires_at: Option<DateTime<Utc>>,
503
504 /// The record this one replaces, when it does. Supersession is a
505 /// chain of immutable records; the superseded record is retained as
506 /// evidence but filtered from default retrieval.
507 #[serde(default, skip_serializing_if = "Option::is_none")]
508 pub supersedes: Option<String>,
509
510 /// Set when the record is a correction-derived candidate pending
511 /// evaluation (R0.8 wave 2; [`Candidacy`]). Additive: absent from the
512 /// wire while unset, and outside the content address (candidacy is
513 /// governance state, not identity).
514 #[serde(default, skip_serializing_if = "Option::is_none")]
515 pub candidacy: Option<Candidacy>,
516
517 /// The record body: inline at or below [`INLINE_PAYLOAD_MAX_BYTES`],
518 /// content-addressed above — the journal's own payload discipline, so
519 /// memory bodies share artifact storage and the large-body story needs
520 /// nothing new.
521 pub content: PayloadRef,
522
523 /// Reserved for vector retrieval (deferred, per the design's not-built
524 /// list). Additive by construction: absent from the wire while unset,
525 /// so the field lands without a wire change when it earns its keep.
526 #[serde(default, skip_serializing_if = "Option::is_none")]
527 pub embedding: Option<Value>,
528}
529
530impl MemoryRecord {
531 /// Build a record, deriving its content address from the content plus
532 /// provenance and splitting the content per the payload discipline
533 /// (inline ≤ [`INLINE_PAYLOAD_MAX_BYTES`], content-addressed above).
534 ///
535 /// Fails when `confidence` is outside `(0, 1]` — the one validity rule
536 /// the contract enforces at construction, because a confidence outside
537 /// the interval is not a claim at all.
538 pub fn new(
539 kind: MemoryKind,
540 scope: ScopeAddress,
541 provenance: MemoryProvenance,
542 confidence: f64,
543 validity: ValidityWindow,
544 created_at: DateTime<Utc>,
545 content: Value,
546 ) -> Result<Self> {
547 if !(confidence > 0.0 && confidence <= 1.0) {
548 return Err(invalid(format!(
549 "memory confidence must be in (0, 1], got {confidence} — confidence is a \
550 writer-declared claim, and a value outside the interval is not a claim at all"
551 )));
552 }
553 let content_ref = content_ref_of(&content);
554 let memory_id = derive_memory_id(&content_ref, &provenance)?;
555 Ok(Self {
556 memory_id,
557 kind,
558 scope,
559 key: None,
560 tags: Vec::new(),
561 priority: 0,
562 provenance,
563 confidence,
564 validity,
565 created_at,
566 expires_at: None,
567 supersedes: None,
568 candidacy: None,
569 content: content_ref,
570 embedding: None,
571 })
572 }
573
574 /// Builder-style: set the lookup key.
575 pub fn with_key(mut self, key: impl Into<String>) -> Self {
576 self.key = Some(key.into());
577 self
578 }
579
580 /// Builder-style: set the tags.
581 pub fn with_tags(mut self, tags: impl IntoIterator<Item = impl Into<String>>) -> Self {
582 self.tags = tags.into_iter().map(Into::into).collect();
583 self
584 }
585
586 /// Builder-style: set the explicit priority (the assembly rank's first
587 /// input).
588 pub fn with_priority(mut self, priority: i64) -> Self {
589 self.priority = priority;
590 self
591 }
592
593 /// Builder-style: set the TTL.
594 pub fn with_expires_at(mut self, expires_at: DateTime<Utc>) -> Self {
595 self.expires_at = Some(expires_at);
596 self
597 }
598
599 /// Builder-style: name the record this one supersedes.
600 pub fn with_supersedes(mut self, supersedes: impl Into<String>) -> Self {
601 self.supersedes = Some(supersedes.into());
602 self
603 }
604
605 /// Builder-style: mark the record as a correction-derived candidate
606 /// (R0.8 wave 2; [`Candidacy`]).
607 pub fn with_candidacy(mut self, candidacy: Candidacy) -> Self {
608 self.candidacy = Some(candidacy);
609 self
610 }
611
612 /// The serialized size of the content in bytes — the input to the
613 /// estimated-token accounting. Inline payloads serialize on demand;
614 /// artifact-referenced payloads carry their size on the reference (the
615 /// canonical-serialization size the address was minted from).
616 pub fn content_bytes(&self) -> u64 {
617 match &self.content {
618 PayloadRef::Inline(value) => serde_json::to_vec(value)
619 .map(|bytes| bytes.len() as u64)
620 .unwrap_or(0),
621 PayloadRef::Artifact(reference) => reference.bytes,
622 }
623 }
624
625 /// Split `content` per the payload discipline, as [`MemoryRecord::new`]
626 /// does: inline at or below [`INLINE_PAYLOAD_MAX_BYTES`],
627 /// content-addressed above. The artifact bytes are the canonical
628 /// serialization of `content`; the caller persists them (the artifact
629 /// store shared with the journal) under the reference's address.
630 pub fn content_ref_of(content: &Value) -> PayloadRef {
631 content_ref_of(content)
632 }
633}
634
635/// The payload-discipline split, free-standing so both the record
636/// constructor and store backends share exactly one decision.
637fn content_ref_of(content: &Value) -> PayloadRef {
638 let Ok(bytes) = serde_json::to_vec(content) else {
639 // A Value that fails to serialize is a contradiction; keep it
640 // inline rather than drop evidence (the journal's own rule).
641 return PayloadRef::Inline(content.clone());
642 };
643 if bytes.len() <= INLINE_PAYLOAD_MAX_BYTES {
644 return PayloadRef::Inline(content.clone());
645 }
646 PayloadRef::Artifact(ArtifactRef {
647 sha256: sha256_hex(&bytes),
648 bytes: bytes.len() as u64,
649 })
650}
651
652/// The content address of a record: `sha256` over the canonical
653/// serialization of `{content, provenance}` — content by hash (so inline
654/// and artifact-referenced forms of the same body address identically) and
655/// provenance in full (origin is identity).
656pub fn derive_memory_id(content: &PayloadRef, provenance: &MemoryProvenance) -> Result<String> {
657 let content_hash = content.content_hash()?;
658 let identity = json!({ "content": content_hash, "provenance": provenance });
659 Ok(sha256_hex(&serde_json::to_vec(&identity)?))
660}
661
662// --------------------------------------------------------------------- //
663// Retrieval: structured filters + context budget
664// --------------------------------------------------------------------- //
665
666/// A structured memory query. Deliberately **not** semantic: R0.8 has no
667/// similarity search (the design's not-built list), so writers key and tag
668/// records deliberately and consumers treat absence of a hit as absence of
669/// a key, not absence of a fact.
670///
671/// Every field is optional; an empty query matches everything in the
672/// namespace (subject to the two defaults: expired and superseded records
673/// are excluded unless explicitly included). The query is journaled as the
674/// read's request, so its serialization is part of the replay contract —
675/// sparse fields stay absent, and [`MemoryQuery::as_of`] is always resolved
676/// before journaling (see [`JournaledMemory::read`]).
677#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
678pub struct MemoryQuery {
679 /// Restrict to one scope address (whose memory, concretely).
680 #[serde(default, skip_serializing_if = "Option::is_none")]
681 pub scope: Option<ScopeAddress>,
682
683 /// Restrict to these kinds (empty = all kinds).
684 #[serde(default, skip_serializing_if = "Vec::is_empty")]
685 pub kinds: Vec<MemoryKind>,
686
687 /// Key equality.
688 #[serde(default, skip_serializing_if = "Option::is_none")]
689 pub key: Option<String>,
690
691 /// Tag equality: a record must carry every listed tag.
692 #[serde(default, skip_serializing_if = "Vec::is_empty")]
693 pub tags: Vec<String>,
694
695 /// Validity-at-time: the record's validity window must contain this
696 /// instant (the bitemporal "true as of" filter).
697 #[serde(default, skip_serializing_if = "Option::is_none")]
698 pub valid_at: Option<DateTime<Utc>>,
699
700 /// Minimum writer-declared confidence, inclusive.
701 #[serde(default, skip_serializing_if = "Option::is_none")]
702 pub min_confidence: Option<f64>,
703
704 /// Include records past their `expires_at` (default: excluded).
705 /// Expiration is a filter, not a reaper — the records are still there.
706 #[serde(default, skip_serializing_if = "is_false")]
707 pub include_expired: bool,
708
709 /// Include records another record supersedes (default: excluded — the
710 /// superseded record is retained as evidence, filtered from default
711 /// retrieval).
712 #[serde(default, skip_serializing_if = "is_false")]
713 pub include_superseded: bool,
714
715 /// Restrict to one author.
716 #[serde(default, skip_serializing_if = "Option::is_none")]
717 pub authored_by: Option<ProvenanceAuthor>,
718
719 /// Restrict to correction-derived candidates pending evaluation (R0.8
720 /// wave 2; default: all records, candidates or not). This is how the
721 /// evaluation half of the loop finds its queue: candidacy is a
722 /// governance mark, so it filters like one — structurally, never by
723 /// content.
724 #[serde(default, skip_serializing_if = "is_false")]
725 pub candidates_only: bool,
726
727 /// The instant expiry is evaluated against. When unset, the reader
728 /// stamps it through the run's clock at read time (wall clock for live
729 /// reads; the deterministic logical clock for recorded runs) — so the
730 /// journaled request always carries the resolved value and a replayed
731 /// read re-derives the same one.
732 #[serde(default, skip_serializing_if = "Option::is_none")]
733 pub as_of: Option<DateTime<Utc>>,
734}
735
736impl MemoryQuery {
737 /// Whether `record` matches every declared filter. `superseded` tells
738 /// whether another record in the queried universe supersedes this one
739 /// (computable only against the whole set, so the caller supplies it);
740 /// `now` is the resolved expiry instant ([`MemoryQuery::as_of`]).
741 pub fn matches(&self, record: &MemoryRecord, superseded: bool, now: DateTime<Utc>) -> bool {
742 if let Some(scope) = &self.scope {
743 if &record.scope != scope {
744 return false;
745 }
746 }
747 if !self.kinds.is_empty() && !self.kinds.contains(&record.kind) {
748 return false;
749 }
750 if let Some(key) = &self.key {
751 if record.key.as_deref() != Some(key.as_str()) {
752 return false;
753 }
754 }
755 if !self
756 .tags
757 .iter()
758 .all(|tag| record.tags.iter().any(|t| t == tag))
759 {
760 return false;
761 }
762 if let Some(valid_at) = self.valid_at {
763 if !record.validity.contains(valid_at) {
764 return false;
765 }
766 }
767 if let Some(min_confidence) = self.min_confidence {
768 if record.confidence < min_confidence {
769 return false;
770 }
771 }
772 if !self.include_expired && record.expires_at.is_some_and(|expires| expires <= now) {
773 return false;
774 }
775 if !self.include_superseded && superseded {
776 return false;
777 }
778 if let Some(author) = &self.authored_by {
779 if &record.provenance.author != author {
780 return false;
781 }
782 }
783 if self.candidates_only && record.candidacy.is_none() {
784 return false;
785 }
786 true
787 }
788}
789
790/// Apply `query` to a universe of records (one tenant's namespace), pure:
791/// the superseded set is computed over the universe, then every declared
792/// filter runs through [`MemoryQuery::matches`]. Both store backends share
793/// this function — the JSON backend scans and applies it directly; Postgres
794/// pre-filters on columns and applies the same function to the reduced set,
795/// so filter semantics live in exactly one place.
796///
797/// The superseded set has two halves. A record is superseded when another
798/// record names it in `supersedes` (the wave-1 rule) — **or, from wave 2,
799/// when a `summary` record names it in
800/// [`MemoryEvidence::source_memory_ids`]**: consolidation supersedes the
801/// records it distills, and the naming is the same one dependent-summary
802/// invalidation walks on forgetting.
803pub fn apply_query(
804 universe: &[MemoryRecord],
805 query: &MemoryQuery,
806 now: DateTime<Utc>,
807) -> Vec<MemoryRecord> {
808 let superseded: HashSet<&str> = superseded_set(universe);
809 universe
810 .iter()
811 .filter(|record| query.matches(record, superseded.contains(record.memory_id.as_str()), now))
812 .cloned()
813 .collect()
814}
815
816/// The tenant namespace's superseded set, as [`apply_query`] defines it:
817/// everything named in a `supersedes` field, plus everything a `summary`
818/// record names as a source (see its docs). Shared by conflict detection
819/// and forget planning, which must agree with retrieval about what is
820/// superseded — three readers, one definition.
821pub fn superseded_set(universe: &[MemoryRecord]) -> HashSet<&str> {
822 universe
823 .iter()
824 .filter_map(|record| record.supersedes.as_deref())
825 .chain(
826 universe
827 .iter()
828 .filter(|record| record.kind == MemoryKind::Summary)
829 .flat_map(|record| {
830 record
831 .provenance
832 .evidence
833 .source_memory_ids
834 .iter()
835 .map(String::as_str)
836 }),
837 )
838 .collect()
839}
840
841/// What the assembly does when the next-ranked record does not fit the
842/// remaining budget.
843#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
844#[serde(rename_all = "snake_case")]
845pub enum BudgetOverflow {
846 /// Stop packing: the assembly carries what fit, ranked, with
847 /// `truncated: true`. The default — a prompt assembly that silently
848 /// drops lower-ranked context is the honest behavior for a budget.
849 #[default]
850 Truncate,
851 /// Fail the read: the caller declared the budget hard and wants to
852 /// know the filtered set did not fit, rather than see a prefix of it.
853 Fail,
854}
855
856/// A prompt-assembly budget, in estimated tokens (see the module docs for
857/// the accounting rule). `ContextBudget` is the seam a model-precise
858/// tokenizer plugs into later: the type stays, the estimate behind
859/// [`estimated_tokens`] is what a later wave may sharpen.
860#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
861pub struct ContextBudget {
862 /// The maximum estimated tokens the assembly may carry.
863 pub max_tokens: u32,
864
865 /// The declared safety margin of the estimate, in percent
866 /// (default [`DEFAULT_TOKEN_MARGIN_PERCENT`]; absent from the wire at
867 /// the default).
868 #[serde(
869 default = "default_margin_percent",
870 skip_serializing_if = "is_default_margin_percent"
871 )]
872 pub margin_percent: u32,
873
874 /// What to do when the next-ranked record does not fit (default
875 /// [`BudgetOverflow::Truncate`]).
876 #[serde(default)]
877 pub overflow: BudgetOverflow,
878}
879
880fn default_margin_percent() -> u32 {
881 DEFAULT_TOKEN_MARGIN_PERCENT
882}
883
884fn is_default_margin_percent(value: &u32) -> bool {
885 *value == DEFAULT_TOKEN_MARGIN_PERCENT
886}
887
888impl ContextBudget {
889 /// A budget of `max_tokens` estimated tokens with the declared default
890 /// margin and truncate overflow.
891 pub fn new(max_tokens: u32) -> Self {
892 Self {
893 max_tokens,
894 margin_percent: DEFAULT_TOKEN_MARGIN_PERCENT,
895 overflow: BudgetOverflow::Truncate,
896 }
897 }
898
899 /// Builder-style: override the safety margin.
900 pub fn with_margin_percent(mut self, margin_percent: u32) -> Self {
901 self.margin_percent = margin_percent;
902 self
903 }
904
905 /// Builder-style: override the overflow policy.
906 pub fn with_overflow(mut self, overflow: BudgetOverflow) -> Self {
907 self.overflow = overflow;
908 self
909 }
910}
911
912/// Estimated tokens for `bytes` of serialized content under the declared
913/// accounting: `ceil(bytes / TOKEN_BYTES_PER_ESTIMATE)` scaled up by the
914/// margin, saturated at `u32::MAX` (an over-budget estimate only means the
915/// record does not fit — the estimate never wraps).
916pub fn estimated_tokens(bytes: u64, margin_percent: u32) -> u32 {
917 let base = bytes.div_ceil(TOKEN_BYTES_PER_ESTIMATE) as u128;
918 let scaled = base * (100 + margin_percent as u128) / 100;
919 scaled.min(u32::MAX as u128) as u32
920}
921
922/// The accounting an assembly was packed under, journaled with it: the
923/// estimate rule (bytes per token and margin), the budget, and how much of
924/// it the assembly used. Recorded — not recomputed — so an auditor reads
925/// the accounting the read actually applied.
926#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
927pub struct TokenAccounting {
928 /// The estimate's divisor ([`TOKEN_BYTES_PER_ESTIMATE`]): estimated
929 /// tokens are serialized content bytes divided by this.
930 pub bytes_per_token: u64,
931
932 /// The safety margin that was applied, in percent.
933 pub margin_percent: u32,
934
935 /// The budget the assembly was packed against, in estimated tokens.
936 pub budget_tokens: u32,
937
938 /// The estimated tokens the packed records sum to.
939 pub used_tokens: u32,
940}
941
942/// The result of a token-bounded retrieval: the filtered records, ranked
943/// and packed. This is the [`RunEventKind::MemoryRead`] event's output
944/// payload — the record ids and their order are first-class
945/// ([`MemoryAssembly::memory_ids`], the design's journaled assembly), and
946/// the full records travel with them so the prompt a model saw is
947/// reconstructable from the journal alone, never from a store query re-run
948/// later against mutated state.
949#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
950pub struct MemoryAssembly {
951 /// The packed records' content addresses, in rank order. The journaled
952 /// fact replay and audit match on.
953 pub memory_ids: Vec<String>,
954
955 /// The packed records, in the same order.
956 pub records: Vec<MemoryRecord>,
957
958 /// The accounting the packing applied.
959 pub token_accounting: TokenAccounting,
960
961 /// `true` when the budget cut the ranked set short (truncate overflow).
962 pub truncated: bool,
963}
964
965/// Rank and pack `records` under `budget`, deterministically.
966///
967/// The rank is a total order — explicit priority descending, then
968/// confidence descending, then recency (`created_at`) descending, then the
969/// content address ascending as the final tie-break — so equal store state
970/// and equal budget produce byte-equal assemblies. Packing walks the ranked
971/// list and stops at the first record that does not fit the remaining
972/// budget ([`BudgetOverflow::Truncate`]) or fails the read
973/// ([`BudgetOverflow::Fail`]).
974pub fn assemble(records: Vec<MemoryRecord>, budget: &ContextBudget) -> Result<MemoryAssembly> {
975 let mut ranked = records;
976 ranked.sort_by(|a, b| {
977 b.priority
978 .cmp(&a.priority)
979 .then_with(|| b.confidence.total_cmp(&a.confidence))
980 .then_with(|| b.created_at.cmp(&a.created_at))
981 .then_with(|| a.memory_id.cmp(&b.memory_id))
982 });
983 let mut used_tokens: u32 = 0;
984 let mut packed: Vec<MemoryRecord> = Vec::new();
985 let mut truncated = false;
986 for record in ranked {
987 let cost = estimated_tokens(record.content_bytes(), budget.margin_percent);
988 if used_tokens.saturating_add(cost) > budget.max_tokens {
989 match budget.overflow {
990 BudgetOverflow::Truncate => {
991 truncated = true;
992 break;
993 }
994 BudgetOverflow::Fail => {
995 return Err(invalid(format!(
996 "memory assembly exceeds the context budget: record `{}` costs an \
997 estimated {cost} tokens with {} of {} already used — the filtered \
998 set does not fit, and the budget was declared hard",
999 record.memory_id, used_tokens, budget.max_tokens
1000 )));
1001 }
1002 }
1003 }
1004 used_tokens = used_tokens.saturating_add(cost);
1005 packed.push(record);
1006 }
1007 let memory_ids = packed
1008 .iter()
1009 .map(|record| record.memory_id.clone())
1010 .collect();
1011 Ok(MemoryAssembly {
1012 memory_ids,
1013 records: packed,
1014 token_accounting: TokenAccounting {
1015 bytes_per_token: TOKEN_BYTES_PER_ESTIMATE,
1016 margin_percent: budget.margin_percent,
1017 budget_tokens: budget.max_tokens,
1018 used_tokens,
1019 },
1020 truncated,
1021 })
1022}
1023
1024// --------------------------------------------------------------------- //
1025// The correction loop (R0.8 wave 2)
1026// --------------------------------------------------------------------- //
1027
1028/// Reject an empty (or all-whitespace) identity at deserialization: a
1029/// correction that cannot name its corrector is indistinguishable from a
1030/// prompt edit, so attribution is enforced where the contract is parsed,
1031/// not where it is consumed.
1032fn deserialize_non_empty_id<'de, D>(deserializer: D) -> std::result::Result<String, D::Error>
1033where
1034 D: serde::Deserializer<'de>,
1035{
1036 let value = String::deserialize(deserializer)?;
1037 if value.trim().is_empty() {
1038 return Err(serde::de::Error::custom(
1039 "identity must be non-empty — attribution is mandatory",
1040 ));
1041 }
1042 Ok(value)
1043}
1044
1045/// What a correction corrects. Closed enum — the three targets the design
1046/// names; consumers match exhaustively.
1047#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
1048#[serde(tag = "type", rename_all = "snake_case")]
1049pub enum CorrectionTarget {
1050 /// A journaled run event (`{run_id}:{seq}`). The correction also yields
1051 /// an `example`-kind record — the input the run saw (read from the
1052 /// journaled event, never re-asked of the world) plus the corrected
1053 /// behavior — which the distiller folds into a versioned dataset: the
1054 /// correction is both the fix and the regression test.
1055 RunEvent {
1056 /// The run whose journal holds the event.
1057 run_id: String,
1058 /// The journaled event's id inside that run.
1059 event_id: String,
1060 },
1061 /// A memory record, by content address. The derived record inherits the
1062 /// target's key, so a same-key correction auto-supersedes the prior
1063 /// record (open question 5).
1064 Memory {
1065 /// The corrected record's content address.
1066 memory_id: String,
1067 },
1068 /// A pinned prompt hash from the run manifest
1069 /// ([`crate::record::RunManifest::pin_prompt`]).
1070 Prompt {
1071 /// The pinned prompt's content hash.
1072 prompt_hash: String,
1073 },
1074}
1075
1076/// A human correction: the highest-trust input the learning system has, and
1077/// the loop treats it accordingly — **a correction becomes an attributed
1078/// candidate memory or example, never an in-place rewrite of what it
1079/// corrects.**
1080///
1081/// `author` is mandatory and validated at deserialization (see
1082/// `deserialize_non_empty_id`); attribution then travels with every
1083/// derived record as `human:{author}` provenance with the correction id in
1084/// evidence — the design's `human:{author} via correction:{id}` string —
1085/// so every later consumer (distiller, evaluator, auditor) can trace the
1086/// record to the person and the moment.
1087#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
1088pub struct Correction {
1089 /// The correction's own id, caller-minted; carried in the derived
1090 /// records' evidence (and therefore their content addresses — two
1091 /// corrections with identical corrected content stay distinct records).
1092 #[serde(deserialize_with = "deserialize_non_empty_id")]
1093 pub correction_id: String,
1094
1095 /// The correcting human's identity. Mandatory.
1096 #[serde(deserialize_with = "deserialize_non_empty_id")]
1097 pub author: String,
1098
1099 /// What it corrects.
1100 pub target: CorrectionTarget,
1101
1102 /// The corrected content: what the derived record asserts.
1103 pub corrected: Value,
1104
1105 /// The scope the result should live at. Scope decides the path
1106 /// ([`Correction::is_candidate`]): run scope is adopted directly (it
1107 /// affects only the run that produced it); agent scope or wider becomes
1108 /// a candidate pending evaluation.
1109 pub scope: ScopeAddress,
1110
1111 /// Why the correction is right, when the corrector said. Optional; the
1112 /// attribution is not.
1113 #[serde(default, skip_serializing_if = "Option::is_none")]
1114 pub rationale: Option<String>,
1115}
1116
1117impl Correction {
1118 /// The derived records' author: `human:{author}`.
1119 pub fn author_as_provenance(&self) -> ProvenanceAuthor {
1120 ProvenanceAuthor::Human {
1121 human_id: self.author.clone(),
1122 }
1123 }
1124
1125 /// The attribution string the design spells: `human:{author} via
1126 /// correction:{id}`. The derived record carries it structurally (author
1127 /// plus evidence); this is the human-facing form, for responses and
1128 /// audit output.
1129 pub fn attribution(&self) -> String {
1130 format!(
1131 "human:{} via correction:{}",
1132 self.author, self.correction_id
1133 )
1134 }
1135
1136 /// The evidence every derived record carries: the correction id, plus
1137 /// the run linkage when the target is a journaled run event.
1138 pub fn evidence(&self) -> MemoryEvidence {
1139 let (run_id, event_ids) = match &self.target {
1140 CorrectionTarget::RunEvent { run_id, event_id } => {
1141 (Some(run_id.clone()), vec![event_id.clone()])
1142 }
1143 _ => (None, Vec::new()),
1144 };
1145 MemoryEvidence {
1146 run_id,
1147 event_ids,
1148 correction_id: Some(self.correction_id.clone()),
1149 ..MemoryEvidence::default()
1150 }
1151 }
1152
1153 /// The scope-decides-the-path rule: `true` when this correction's
1154 /// derived records are candidates pending evaluation (agent scope or
1155 /// wider), `false` at run scope, where a correction is adopted directly
1156 /// — it affects only the run that produced it.
1157 pub fn is_candidate(&self) -> bool {
1158 self.scope.scope != MemoryScope::Run
1159 }
1160}
1161
1162// --------------------------------------------------------------------- //
1163// Forgetting (R0.8 wave 2): real deletion with a journaled receipt
1164// --------------------------------------------------------------------- //
1165
1166/// Why a record was forgotten. Closed enum, carried on the tombstone: the
1167/// reason is what distinguishes a reaper's expiry from a GDPR-shaped
1168/// erasure request in the audit trail.
1169#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
1170#[serde(rename_all = "snake_case")]
1171pub enum ForgetReason {
1172 /// The record's TTL lapsed. Expiration stays a retrieval filter too —
1173 /// this is the operator-driven sweep acting on it, journaled.
1174 Expired,
1175 /// The record was wrong (or its correction superseded the need for it)
1176 /// and is withdrawn.
1177 Retracted,
1178 /// An erasure request (a user's, an operator's compliance pass).
1179 ErasureRequest,
1180}
1181
1182/// The journaled receipt of a forgetting — the
1183/// [`RunEventKind::MemoryForget`] event's output payload. **Metadata by
1184/// construction**: the forgotten id, its scope, the reason, and the
1185/// dependent invalidations. There is no content field to leave unset — a
1186/// tombstone that *could* carry the forgotten bytes would be one careless
1187/// serializer away from defeating the erasure it receipts.
1188#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
1189pub struct MemoryForgetTombstone {
1190 /// The forgotten record's content address.
1191 pub memory_id: String,
1192
1193 /// The scope the record lived at.
1194 pub scope: ScopeAddress,
1195
1196 /// Why it was forgotten.
1197 pub reason: ForgetReason,
1198
1199 /// The dependent summaries the erasure invalidated (the reverse walk
1200 /// over source naming, transitively — see [`plan_forget`]). Their ids,
1201 /// never their content.
1202 #[serde(default, skip_serializing_if = "Vec::is_empty")]
1203 pub invalidated: Vec<String>,
1204}
1205
1206/// The derived idempotency key of a forgetting:
1207/// `memory_forget:{scope}:{memory_id}`. Retried erasures converge on one
1208/// key — forgetting an already-forgotten record is the same journaled
1209/// effect, not a second one.
1210pub fn memory_forget_effect_key(scope: &ScopeAddress, memory_id: &str) -> String {
1211 format!("memory_forget:{}:{memory_id}", scope.as_address())
1212}
1213
1214/// What a forgetting will do, computed over the namespace **before**
1215/// anything is deleted: the explicitly forgotten records and the dependent
1216/// summaries the erasure invalidates. Planning is a pure function of store
1217/// state, so the route can journal exactly what it is about to do — the
1218/// tombstone names this plan, and the deletion then executes it.
1219#[derive(Debug, Clone, Default, PartialEq, Eq)]
1220pub struct ForgetPlan {
1221 /// The explicitly forgotten records (the targets that exist in the
1222 /// namespace), sorted for determinism.
1223 pub forgotten: Vec<String>,
1224
1225 /// The dependent summaries invalidated by the erasure, sorted: every
1226 /// `summary` record that names a removed record in
1227 /// [`MemoryEvidence::source_memory_ids`], transitively (a summary of
1228 /// summaries goes too). They are deleted with the targets — a summary
1229 /// built on erased evidence is derived state, and leaving it serving
1230 /// content distilled from the forgotten record is not forgetting
1231 /// (Cao & Yang's point, per the design).
1232 pub invalidated: Vec<String>,
1233}
1234
1235/// Plan a forgetting over `universe` (one tenant's namespace): `targets`
1236/// are the records to erase; the dependent-summary walk runs in reverse
1237/// over source naming until the closure stops growing. Targets absent from
1238/// the universe are skipped (the caller decides whether absence is an
1239/// error — the single-id route 404s before planning).
1240pub fn plan_forget(universe: &[MemoryRecord], targets: &[String]) -> ForgetPlan {
1241 let mut removed: HashSet<&str> = universe
1242 .iter()
1243 .filter(|record| targets.iter().any(|t| t == &record.memory_id))
1244 .map(|record| record.memory_id.as_str())
1245 .collect();
1246 // The fixpoint: a summary joins the removal set whenever it names a
1247 // record already in it. Terminates because the set grows monotonically
1248 // over a finite universe.
1249 loop {
1250 let mut grew = false;
1251 for record in universe {
1252 if record.kind != MemoryKind::Summary || removed.contains(record.memory_id.as_str()) {
1253 continue;
1254 }
1255 if record
1256 .provenance
1257 .evidence
1258 .source_memory_ids
1259 .iter()
1260 .any(|source| removed.contains(source.as_str()))
1261 {
1262 removed.insert(record.memory_id.as_str());
1263 grew = true;
1264 }
1265 }
1266 if !grew {
1267 break;
1268 }
1269 }
1270 let mut forgotten: Vec<String> = removed
1271 .iter()
1272 .filter(|id| targets.iter().any(|t| t.as_str() == **id))
1273 .map(|id| id.to_string())
1274 .collect();
1275 forgotten.sort();
1276 let mut invalidated: Vec<String> = removed
1277 .iter()
1278 .filter(|id| !forgotten.iter().any(|f| f.as_str() == **id))
1279 .map(|id| id.to_string())
1280 .collect();
1281 invalidated.sort();
1282 ForgetPlan {
1283 forgotten,
1284 invalidated,
1285 }
1286}
1287
1288// --------------------------------------------------------------------- //
1289// Conflict detection (R0.8 wave 2): evidence, never resolution
1290// --------------------------------------------------------------------- //
1291
1292/// A flagged conflict: two live records that share a key, overlap in
1293/// validity, and carry contradictory content. A review item — the design's
1294/// rule is that detection is evidence and resolution is governance, so
1295/// nothing here (or anywhere in the runtime) resolves the pair: Zep and
1296/// Mem0 resolve contradictions inside the ingestion pipeline with an LLM,
1297/// which is precisely the silent-mutation pattern the learning rule exists
1298/// to forbid.
1299#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
1300pub struct MemoryConflict {
1301 /// The scope the conflicted records live at.
1302 pub scope: ScopeAddress,
1303
1304 /// The key they share.
1305 pub key: String,
1306
1307 /// The two records' content addresses, sorted.
1308 pub memory_ids: Vec<String>,
1309
1310 /// The interval both records claim to be true over — the overlap of
1311 /// their validity windows.
1312 pub overlap: ValidityWindow,
1313}
1314
1315/// Flag the conflicts in `universe` (one tenant's namespace) at `now`.
1316///
1317/// "Live" excludes exactly what default retrieval excludes: superseded
1318/// records (a superseded record is replaced evidence, not a contender) and
1319/// records expired at `now`. "Contradictory" is structural and honest about
1320/// it: with no semantic judge in the runtime, contradiction means the two
1321/// records assert *different canonical content* for the same key over an
1322/// overlapping interval — semantics belong to the reviewer the flag is for.
1323/// A pair where one record supersedes the other is disciplined replacement,
1324/// not conflict. The output is deterministically ordered (scope, key, ids),
1325/// so equal store state flags byte-equal review items.
1326pub fn detect_conflicts(universe: &[MemoryRecord], now: DateTime<Utc>) -> Vec<MemoryConflict> {
1327 let superseded = superseded_set(universe);
1328 let is_live = |record: &MemoryRecord| {
1329 !superseded.contains(record.memory_id.as_str())
1330 && record.expires_at.is_none_or(|expires| expires > now)
1331 };
1332 let mut groups: BTreeMap<(String, &str), Vec<&MemoryRecord>> = BTreeMap::new();
1333 for record in universe.iter().filter(|r| is_live(r)) {
1334 let Some(key) = record.key.as_deref() else {
1335 continue;
1336 };
1337 groups
1338 .entry((record.scope.as_address(), key))
1339 .or_default()
1340 .push(record);
1341 }
1342 let mut conflicts = Vec::new();
1343 for ((_, key), mut group) in groups {
1344 group.sort_by(|a, b| a.memory_id.cmp(&b.memory_id));
1345 for (i, a) in group.iter().enumerate() {
1346 for b in &group[i + 1..] {
1347 if a.supersedes.as_deref() == Some(b.memory_id.as_str())
1348 || b.supersedes.as_deref() == Some(a.memory_id.as_str())
1349 {
1350 continue;
1351 }
1352 let Some(overlap) = validity_overlap(&a.validity, &b.validity) else {
1353 continue;
1354 };
1355 if same_content(a, b) {
1356 continue;
1357 }
1358 conflicts.push(MemoryConflict {
1359 scope: a.scope.clone(),
1360 key: key.to_string(),
1361 memory_ids: vec![a.memory_id.clone(), b.memory_id.clone()],
1362 overlap,
1363 });
1364 }
1365 }
1366 }
1367 conflicts.sort_by(|a, b| {
1368 a.scope
1369 .as_address()
1370 .cmp(&b.scope.as_address())
1371 .then_with(|| a.key.cmp(&b.key))
1372 .then_with(|| a.memory_ids.cmp(&b.memory_ids))
1373 });
1374 conflicts
1375}
1376
1377/// The intersection of two validity windows: `[max(from), min(until))`,
1378/// `None` when the windows are disjoint. Open-endedness composes (open ∩
1379/// closed is the closed end; open ∩ open stays open).
1380fn validity_overlap(a: &ValidityWindow, b: &ValidityWindow) -> Option<ValidityWindow> {
1381 let valid_from = a.valid_from.max(b.valid_from);
1382 let valid_until = match (a.valid_until, b.valid_until) {
1383 (Some(x), Some(y)) => Some(x.min(y)),
1384 (x, y) => x.or(y),
1385 };
1386 if valid_until.is_some_and(|until| valid_from >= until) {
1387 return None;
1388 }
1389 Some(ValidityWindow {
1390 valid_from,
1391 valid_until,
1392 })
1393}
1394
1395/// Content equality by canonical hash, so inline and artifact-referenced
1396/// forms of the same body compare equal (the payload discipline's rule
1397/// applied to conflict detection).
1398fn same_content(a: &MemoryRecord, b: &MemoryRecord) -> bool {
1399 match (a.content.content_hash(), b.content.content_hash()) {
1400 (Ok(x), Ok(y)) => x == y,
1401 _ => a.content == b.content,
1402 }
1403}
1404
1405// --------------------------------------------------------------------- //
1406// Consolidation (R0.8 wave 2): the runtime-owned summary invariants
1407// --------------------------------------------------------------------- //
1408
1409/// Build the one `summary` record that distills `sources`, under the
1410/// invariants the runtime owns: the sources are named in
1411/// [`MemoryEvidence::source_memory_ids`] (which supersedes them — see
1412/// [`apply_query`] — and makes dependent-summary invalidation computable on
1413/// forgetting), the author is `distiller:{name}`, and the confidence is the
1414/// **minimum of the sources'**: a summary is a claim no stronger than its
1415/// weakest source, and the model is honest about that.
1416///
1417/// What the runtime deliberately does not own: `content`. The distillation
1418/// semantics are application code (open question 2's boundary, the same one
1419/// the distiller contract draws); this function is the orchestration around
1420/// them. The record's validity spans the sources (earliest `valid_from`;
1421/// the latest `valid_until` when every source is closed, open-ended
1422/// otherwise — a summary cannot outclaim the evidence it distills).
1423///
1424/// Fails when `sources` is empty: a summary that names no sources is not a
1425/// consolidation, and an unnamed derivation cannot be audited or forgotten
1426/// correctly.
1427pub fn consolidation_summary(
1428 scope: ScopeAddress,
1429 distiller: impl Into<String>,
1430 sources: &[MemoryRecord],
1431 content: Value,
1432 now: DateTime<Utc>,
1433) -> Result<MemoryRecord> {
1434 if sources.is_empty() {
1435 return Err(invalid(
1436 "consolidation needs at least one source record — a summary that names no \
1437 sources is not a consolidation, and an unnamed derivation can neither be \
1438 audited nor forgotten correctly",
1439 ));
1440 }
1441 let mut source_ids: Vec<String> = sources
1442 .iter()
1443 .map(|record| record.memory_id.clone())
1444 .collect();
1445 source_ids.sort();
1446 let confidence = sources
1447 .iter()
1448 .map(|record| record.confidence)
1449 .fold(f64::INFINITY, f64::min);
1450 let valid_from = sources
1451 .iter()
1452 .map(|record| record.validity.valid_from)
1453 .min()
1454 .expect("sources is non-empty");
1455 let valid_until = sources
1456 .iter()
1457 .map(|record| record.validity.valid_until)
1458 .collect::<Option<Vec<DateTime<Utc>>>>()
1459 .and_then(|untils| untils.into_iter().max());
1460 let provenance = MemoryProvenance {
1461 author: ProvenanceAuthor::Distiller {
1462 name: distiller.into(),
1463 },
1464 evidence: MemoryEvidence {
1465 source_memory_ids: source_ids,
1466 ..MemoryEvidence::default()
1467 },
1468 written_at: now,
1469 };
1470 MemoryRecord::new(
1471 MemoryKind::Summary,
1472 scope,
1473 provenance,
1474 confidence,
1475 ValidityWindow {
1476 valid_from,
1477 valid_until,
1478 },
1479 now,
1480 content,
1481 )
1482}
1483
1484// --------------------------------------------------------------------- //
1485// The store contract
1486// --------------------------------------------------------------------- //
1487
1488/// A governed memory store. The two server backends (JSON files, Postgres)
1489/// implement it over their own layouts; [`InMemoryMemoryStore`] is the
1490/// dev/test implementation, the same role `InMemoryCheckpointer` plays for
1491/// checkpoints.
1492///
1493/// The contract deliberately knows nothing about tenants: tenant isolation
1494/// is the server's `{tenant}/` id-namespacing, applied by the server
1495/// backends around this contract — a store that can see another tenant's
1496/// records is a breach wearing a storage costume.
1497#[async_trait]
1498pub trait MemoryStore: Send + Sync + std::fmt::Debug {
1499 /// Store `record` under its content address. Idempotent: storing the
1500 /// same record twice yields `false` (already present) and at most one
1501 /// stored record — the `Effect::Idempotent` write converges by
1502 /// construction, since the address is derived from content plus
1503 /// provenance.
1504 async fn put(&self, record: &MemoryRecord) -> Result<bool>;
1505
1506 /// Fetch one record by content address (`None` when absent).
1507 async fn get(&self, memory_id: &str) -> Result<Option<MemoryRecord>>;
1508
1509 /// The query universe: every record in the namespace. Retrieval
1510 /// semantics live in [`apply_query`]; dev-scale stores scan, and larger
1511 /// backends pre-filter on columns before applying the same function.
1512 async fn all(&self) -> Result<Vec<MemoryRecord>>;
1513
1514 /// Remove `memory_id` from the store (`false` when absent). Forgetting
1515 /// (R0.8 wave 2) is real deletion of derived state — journals are
1516 /// hash-chained evidence and are never touched (open question 4's
1517 /// boundary).
1518 async fn remove(&self, memory_id: &str) -> Result<bool>;
1519
1520 /// The records matching `query`, expiry evaluated at `now`. The default
1521 /// implementation scans via [`MemoryStore::all`] and applies
1522 /// [`apply_query`]; backends with column-mapped storage override the
1523 /// scan, never the semantics.
1524 async fn query(&self, query: &MemoryQuery, now: DateTime<Utc>) -> Result<Vec<MemoryRecord>> {
1525 Ok(apply_query(&self.all().await?, query, now))
1526 }
1527}
1528
1529/// In-memory [`MemoryStore`] (dev and test): one map keyed by content
1530/// address. No persistence, no tenant concept — the honest reference for
1531/// the contract's semantics.
1532#[derive(Debug, Default)]
1533pub struct InMemoryMemoryStore {
1534 records: Mutex<BTreeMap<String, MemoryRecord>>,
1535}
1536
1537impl InMemoryMemoryStore {
1538 /// An empty store.
1539 pub fn new() -> Self {
1540 Self::default()
1541 }
1542
1543 fn lock(&self) -> MutexGuard<'_, BTreeMap<String, MemoryRecord>> {
1544 // Poison means a store path panicked mid-write; the map is plain
1545 // data and stays coherent, so recovering is safe.
1546 self.records.lock().unwrap_or_else(|e| e.into_inner())
1547 }
1548}
1549
1550#[async_trait]
1551impl MemoryStore for InMemoryMemoryStore {
1552 async fn put(&self, record: &MemoryRecord) -> Result<bool> {
1553 let mut map = self.lock();
1554 if map.contains_key(&record.memory_id) {
1555 return Ok(false);
1556 }
1557 map.insert(record.memory_id.clone(), record.clone());
1558 Ok(true)
1559 }
1560
1561 async fn get(&self, memory_id: &str) -> Result<Option<MemoryRecord>> {
1562 Ok(self.lock().get(memory_id).cloned())
1563 }
1564
1565 async fn all(&self) -> Result<Vec<MemoryRecord>> {
1566 Ok(self.lock().values().cloned().collect())
1567 }
1568
1569 async fn remove(&self, memory_id: &str) -> Result<bool> {
1570 Ok(self.lock().remove(memory_id).is_some())
1571 }
1572}
1573
1574// --------------------------------------------------------------------- //
1575// The journaled seam
1576// --------------------------------------------------------------------- //
1577
1578/// The canonical JSON shape a [`JournaledMemory`] read journals (and a
1579/// [`MemoryReplaySource`] matches) as a memory read's **request**: the
1580/// resolved query (with [`MemoryQuery::as_of`] stamped) and the budget,
1581/// exactly as passed to the store. The hash of this value's canonical
1582/// serialization is the request identity exact replay matches on — the same
1583/// rule [`crate::replay::model_call_request`] sets for model calls.
1584pub fn memory_read_request(query: &MemoryQuery, budget: &ContextBudget) -> Value {
1585 json!({ "query": query, "budget": budget })
1586}
1587
1588/// The derived idempotency key of a memory write:
1589/// `memory:{scope}:{memory_id}` (the design's write-path rule). Retried
1590/// submissions of the same write converge on one key, and recovery can
1591/// re-derive it without consulting the store.
1592pub fn memory_effect_key(scope: &ScopeAddress, memory_id: &str) -> String {
1593 format!("memory:{}:{memory_id}", scope.as_address())
1594}
1595
1596/// Canonical-content hash of a request value — the same computation
1597/// `PayloadRef::content_hash` applies and the same one
1598/// [`crate::replay::ReplaySource`] matches model/tool requests with.
1599fn request_hash(request: &Value) -> Result<String> {
1600 Ok(sha256_hex(&serde_json::to_vec(request)?))
1601}
1602
1603#[derive(Debug)]
1604struct MemoryReplayInner {
1605 /// The snapshot's `MemoryRead` events, in `seq` order.
1606 servable: Vec<RunEvent>,
1607 /// The snapshot's artifact map, for payload resolution.
1608 artifacts: BTreeMap<String, Value>,
1609 /// Position of the next unserved event. Strictly ordered, the same rule
1610 /// the Flight Recorder's replay source applies.
1611 cursor: usize,
1612}
1613
1614/// The serving side of exact replay for memory reads: an ordered cursor
1615/// over a journaled run's recorded [`RunEventKind::MemoryRead`] events.
1616///
1617/// This is [`crate::replay::ReplaySource`] specialized to one event kind —
1618/// the same matching rule (sequence + request hash), the same loud failure
1619/// on any mismatch — kept in this module because a memory read is answered
1620/// from the memory seam, not from a model or tool wrapper. Matching is by
1621/// **sequence + request hash**: [`MemoryReplaySource::serve`] takes the next
1622/// unserved `MemoryRead` event in `seq` order and requires the issued
1623/// request's canonical hash to equal the journaled request's. Anything else
1624/// fails with [`RustyError::Replay`] — a replay that has drifted from its
1625/// evidence must stop, not improvise.
1626#[derive(Debug, Clone)]
1627pub struct MemoryReplaySource {
1628 inner: Arc<Mutex<MemoryReplayInner>>,
1629}
1630
1631impl MemoryReplaySource {
1632 /// A source over the `MemoryRead` events of `snapshot`. The snapshot is
1633 /// assumed already integrity-verified (the caller's replay session
1634 /// verifies it); unresolved artifact references simply fail to match.
1635 pub fn new(snapshot: &JournalSnapshot) -> Self {
1636 Self {
1637 inner: Arc::new(Mutex::new(MemoryReplayInner {
1638 servable: snapshot
1639 .events
1640 .iter()
1641 .filter(|event| event.kind == RunEventKind::MemoryRead)
1642 .cloned()
1643 .collect(),
1644 artifacts: snapshot.artifacts.clone(),
1645 cursor: 0,
1646 })),
1647 }
1648 }
1649
1650 fn lock(&self) -> MutexGuard<'_, MemoryReplayInner> {
1651 self.inner.lock().unwrap_or_else(|e| e.into_inner())
1652 }
1653
1654 /// Serve one memory read from the journal: the next unserved
1655 /// `MemoryRead` event whose journaled request hash-equals `request`.
1656 /// The returned [`ServedEffect`] carries the recorded assembly; re-journal
1657 /// it into the replay run's journal so the replayed evidence reproduces
1658 /// the recorded evidence byte-for-byte.
1659 pub fn serve(&self, request: &Value) -> Result<ServedEffect> {
1660 let issued_hash = request_hash(request)?;
1661 let mut inner = self.lock();
1662 let Some(event) = inner.servable.get(inner.cursor) else {
1663 return Err(replay_error(format!(
1664 "replay exhaustion: the run issued a memory read (request hash {issued_hash}), \
1665 but the journal holds no further recorded memory reads to serve — the \
1666 replayed run is doing more work than the recorded one"
1667 )));
1668 };
1669 let recorded_input = event
1670 .input
1671 .as_ref()
1672 .and_then(|payload| match payload {
1673 PayloadRef::Inline(value) => Some(value.clone()),
1674 PayloadRef::Artifact(reference) => inner.artifacts.get(&reference.sha256).cloned(),
1675 })
1676 .ok_or_else(|| {
1677 replay_error(format!(
1678 "recorded memory read at seq {} has no input payload to match against",
1679 event.seq
1680 ))
1681 })?;
1682 let recorded_hash = request_hash(&recorded_input)?;
1683 if recorded_hash != issued_hash {
1684 return Err(replay_error(format!(
1685 "replay divergence at recorded seq {} (memory read): the run issued a request \
1686 whose canonical hash is {issued_hash}, but the journaled request hashes to \
1687 {recorded_hash} — the replayed run has diverged from its evidence",
1688 event.seq
1689 )));
1690 }
1691 let served = ServedEffect {
1692 input: Some(recorded_input),
1693 output: event.output.as_ref().and_then(|payload| match payload {
1694 PayloadRef::Inline(value) => Some(value.clone()),
1695 PayloadRef::Artifact(reference) => inner.artifacts.get(&reference.sha256).cloned(),
1696 }),
1697 event: event.clone(),
1698 };
1699 inner.cursor += 1;
1700 Ok(served)
1701 }
1702
1703 /// `true` when every recorded memory read has been served. Verification
1704 /// treats a non-exhausted source as a replay that stopped short of its
1705 /// evidence.
1706 pub fn is_exhausted(&self) -> bool {
1707 let inner = self.lock();
1708 inner.cursor == inner.servable.len()
1709 }
1710}
1711
1712/// Where a journaled memory handle answers reads from: the live store
1713/// (record mode) or a recorded journal (replay mode).
1714///
1715/// Node code holds one [`JournaledMemory`] either way and cannot tell the
1716/// difference — the same discipline the Flight Recorder's recording and
1717/// replaying model wrappers apply: behavior that matters is journaled, and
1718/// replay serves the journal instead of the world.
1719#[derive(Debug, Clone)]
1720pub enum MemorySource {
1721 /// Read from and write to a live [`MemoryStore`], journaling every
1722 /// operation.
1723 Store(Arc<dyn MemoryStore>),
1724
1725 /// Read from a recorded journal ([`MemoryReplaySource`]), re-journaling
1726 /// the served events byte-identically. Writes fail: a replayed run
1727 /// issuing a write has diverged from its evidence.
1728 Replay(MemoryReplaySource),
1729}
1730
1731/// A memory handle bound to a run's journal: the journaled read/write seam
1732/// node code (and later waves' run integration) uses.
1733///
1734/// Cheap to clone (the journal and source are shared handles), so node
1735/// closures capture it exactly the way they capture a [`Journal`] for
1736/// recording model and tool calls. Causal parentage comes from the
1737/// invocation's node-input event id, passed as `parent` — node code reads
1738/// it from [`crate::journal::PARENT_EVENT_KEY`] in `NodeConfig::extra`.
1739///
1740/// Clock-read parity: the live and replay paths read the run's clock the
1741/// same number of times in the same order (one `as_of` stamp, two latency
1742/// reads, one read inside [`Journal::record`]), so a logical clock's tick
1743/// sequence — and therefore every journaled timestamp — reproduces exactly
1744/// on replay. This is the same parity rule the replaying model wrapper
1745/// documents.
1746#[derive(Debug, Clone)]
1747pub struct JournaledMemory {
1748 journal: Journal,
1749 source: MemorySource,
1750}
1751
1752impl JournaledMemory {
1753 /// Bind `source` to `journal` (the run's journal — build via
1754 /// [`Journal::memory`]).
1755 pub fn new(journal: &Journal, source: MemorySource) -> Self {
1756 Self {
1757 journal: journal.clone(),
1758 source,
1759 }
1760 }
1761
1762 /// The source this handle answers from.
1763 pub fn source(&self) -> &MemorySource {
1764 &self.source
1765 }
1766
1767 /// Perform a governed read: resolve [`MemoryQuery::as_of`] through the
1768 /// run's clock when unset, then — live — query the store, assemble
1769 /// deterministically, and journal the assembly as a
1770 /// [`RunEventKind::MemoryRead`] event ([`Effect::ReadOnly`]); or —
1771 /// replay — serve the journaled assembly byte-identically and re-journal
1772 /// it. Returns the assembly either way.
1773 pub async fn read(
1774 &self,
1775 query: &MemoryQuery,
1776 budget: &ContextBudget,
1777 parent: Option<String>,
1778 ) -> Result<MemoryAssembly> {
1779 // The resolved query fully determines the result: as_of stamped
1780 // through the run's clock (deterministic under a logical clock),
1781 // everything else declared. One clock read here in both paths —
1782 // the parity the byte-identical replay property rests on.
1783 let mut resolved = query.clone();
1784 let as_of = resolved.as_of.unwrap_or_else(|| self.journal.clock().now());
1785 resolved.as_of = Some(as_of);
1786 let request = memory_read_request(&resolved, budget);
1787 match &self.source {
1788 MemorySource::Store(store) => {
1789 let started = self.journal.clock().now();
1790 let records = store.query(&resolved, as_of).await?;
1791 let latency_ms = (self.journal.clock().now() - started)
1792 .num_milliseconds()
1793 .max(0) as u64;
1794 let assembly = assemble(records, budget)?;
1795 let mut draft = EventDraft::new(RunEventKind::MemoryRead, Effect::ReadOnly)
1796 .input(request)
1797 .output(serde_json::to_value(&assembly)?)
1798 .latency_ms(latency_ms);
1799 if let Some(parent) = parent {
1800 draft = draft.parent(parent);
1801 }
1802 self.journal.record(draft);
1803 Ok(assembly)
1804 }
1805 MemorySource::Replay(source) => {
1806 let served = source.serve(&request)?;
1807 // Clock-read parity with the live path (two latency reads
1808 // per read) keeps the logical clock's tick sequence aligned
1809 // with the recorded run, so the replayed journal's
1810 // timestamps reproduce it exactly.
1811 let _started = self.journal.clock().now();
1812 let _ended = self.journal.clock().now();
1813 let parent = parent
1814 .or_else(|| served.event.parent.clone())
1815 .unwrap_or_default();
1816 served.rejournal(&self.journal, parent);
1817 let output = served.output.clone().ok_or_else(|| {
1818 replay_error(format!(
1819 "recorded memory read at seq {} carries no assembly payload — the \
1820 journal is inconsistent",
1821 served.event.seq
1822 ))
1823 })?;
1824 let assembly: MemoryAssembly = serde_json::from_value(output)?;
1825 Ok(assembly)
1826 }
1827 }
1828 }
1829
1830 /// Perform a governed write: store the record (content-addressed, so
1831 /// retries converge) and journal it as a [`RunEventKind::MemoryWrite`]
1832 /// event — [`Effect::Idempotent`] under the derived key
1833 /// `memory:{scope}:{memory_id}`, with the record and its provenance as
1834 /// the output payload. Returns the content address.
1835 ///
1836 /// A memory write changes what future retrievals return — nothing else
1837 /// (the write path's third gate). If the write is meant to change a
1838 /// prompt, a policy, or a permission, it belongs in the candidate
1839 /// pipeline, not here.
1840 ///
1841 /// Fails during replay: writes are never served, so a replayed run
1842 /// issuing one has diverged from its evidence.
1843 pub async fn write(&self, record: &MemoryRecord, parent: Option<String>) -> Result<String> {
1844 match &self.source {
1845 MemorySource::Store(store) => {
1846 store.put(record).await?;
1847 let effect_key = memory_effect_key(&record.scope, &record.memory_id);
1848 let mut draft = EventDraft::new(RunEventKind::MemoryWrite, Effect::Idempotent)
1849 .input(json!({
1850 "effect_key": effect_key,
1851 "memory_id": record.memory_id,
1852 }))
1853 .output(serde_json::to_value(record)?);
1854 if let Some(parent) = parent {
1855 draft = draft.parent(parent);
1856 }
1857 self.journal.record(draft);
1858 Ok(record.memory_id.clone())
1859 }
1860 MemorySource::Replay(_) => Err(replay_error(
1861 "memory writes are not served during replay: the run issued a write the \
1862 recorded evidence does not contain — the replayed run has diverged from its \
1863 evidence",
1864 )),
1865 }
1866 }
1867}
1868
1869#[cfg(test)]
1870mod tests {
1871 use super::*;
1872 use crate::journal::Clock;
1873
1874 fn ts(millis: i64) -> DateTime<Utc> {
1875 DateTime::<Utc>::from_timestamp_millis(millis).unwrap()
1876 }
1877
1878 fn provenance() -> MemoryProvenance {
1879 MemoryProvenance {
1880 author: ProvenanceAuthor::Human {
1881 human_id: "amjad".into(),
1882 },
1883 evidence: MemoryEvidence::default(),
1884 written_at: ts(1_000),
1885 }
1886 }
1887
1888 fn record(content: Value) -> MemoryRecord {
1889 MemoryRecord::new(
1890 MemoryKind::Fact,
1891 ScopeAddress::new(MemoryScope::User, "user-7"),
1892 provenance(),
1893 1.0,
1894 ValidityWindow::starting(ts(500)),
1895 ts(1_000),
1896 content,
1897 )
1898 .unwrap()
1899 }
1900
1901 #[test]
1902 fn scope_mapping_is_the_documented_superset() {
1903 assert_eq!(
1904 MemoryScope::from_state_scope(StateScope::Private),
1905 Some(MemoryScope::Agent)
1906 );
1907 assert_eq!(
1908 MemoryScope::from_state_scope(StateScope::Team),
1909 Some(MemoryScope::Team)
1910 );
1911 assert_eq!(
1912 MemoryScope::from_state_scope(StateScope::User),
1913 Some(MemoryScope::User)
1914 );
1915 assert_eq!(
1916 MemoryScope::from_state_scope(StateScope::Tenant),
1917 Some(MemoryScope::Tenant)
1918 );
1919 assert_eq!(
1920 MemoryScope::Agent.to_state_scope(),
1921 Some(StateScope::Private)
1922 );
1923 assert_eq!(MemoryScope::Run.to_state_scope(), None);
1924 for scope in [
1925 MemoryScope::Run,
1926 MemoryScope::Agent,
1927 MemoryScope::Team,
1928 MemoryScope::User,
1929 MemoryScope::Tenant,
1930 ] {
1931 if let Some(state_scope) = scope.to_state_scope() {
1932 assert_eq!(MemoryScope::from_state_scope(state_scope), Some(scope));
1933 }
1934 }
1935 }
1936
1937 #[test]
1938 fn memory_id_is_a_content_address() {
1939 let a = record(json!({"timezone": "UTC+4"}));
1940 let b = record(json!({"timezone": "UTC+4"}));
1941 let c = record(json!({"timezone": "UTC+1"}));
1942 // Identical content and provenance converge on one id…
1943 assert_eq!(a.memory_id, b.memory_id);
1944 assert_eq!(a.memory_id.len(), 64);
1945 // …changed content is a new id…
1946 assert_ne!(a.memory_id, c.memory_id);
1947 // …and so is changed provenance (origin is identity).
1948 let mut other_provenance = provenance();
1949 other_provenance.written_at = ts(2_000);
1950 let d = MemoryRecord::new(
1951 MemoryKind::Fact,
1952 ScopeAddress::new(MemoryScope::User, "user-7"),
1953 other_provenance,
1954 1.0,
1955 ValidityWindow::starting(ts(500)),
1956 ts(1_000),
1957 json!({"timezone": "UTC+4"}),
1958 )
1959 .unwrap();
1960 assert_ne!(a.memory_id, d.memory_id);
1961 }
1962
1963 #[test]
1964 fn confidence_outside_the_interval_is_rejected() {
1965 for confidence in [0.0, -0.1, 1.01, f64::NAN] {
1966 assert!(
1967 MemoryRecord::new(
1968 MemoryKind::Fact,
1969 ScopeAddress::new(MemoryScope::User, "u"),
1970 provenance(),
1971 confidence,
1972 ValidityWindow::starting(ts(0)),
1973 ts(0),
1974 json!(null),
1975 )
1976 .is_err(),
1977 "confidence {confidence} must be rejected"
1978 );
1979 }
1980 }
1981
1982 #[test]
1983 fn content_split_follows_the_payload_discipline() {
1984 let small = record(json!({"k": "v"}));
1985 assert!(matches!(small.content, PayloadRef::Inline(_)));
1986 let big = record(json!({"blob": "x".repeat(INLINE_PAYLOAD_MAX_BYTES)}));
1987 let reference = match &big.content {
1988 PayloadRef::Artifact(reference) => reference.clone(),
1989 other => panic!("expected artifact reference, got {other:?}"),
1990 };
1991 assert_eq!(reference.bytes, big.content_bytes());
1992 // The address is derivable from the record itself: identity is
1993 // integrity.
1994 assert_eq!(
1995 derive_memory_id(&big.content, &big.provenance).unwrap(),
1996 big.memory_id
1997 );
1998 }
1999
2000 #[test]
2001 fn estimated_tokens_applies_bytes_divisor_and_margin() {
2002 assert_eq!(estimated_tokens(0, 20), 0);
2003 assert_eq!(estimated_tokens(4, 0), 1);
2004 assert_eq!(estimated_tokens(5, 0), 2); // ceil
2005 assert_eq!(estimated_tokens(400, 20), 120); // 100 * 1.2
2006 assert_eq!(estimated_tokens(u64::MAX, 100), u32::MAX); // saturated
2007 }
2008
2009 #[tokio::test]
2010 async fn journaled_write_is_idempotent_under_the_derived_key() {
2011 let journal = Journal::new("run-1", "thread-1", Clock::System);
2012 let store = Arc::new(InMemoryMemoryStore::new());
2013 let memory = journal.memory(MemorySource::Store(store.clone()));
2014 let record = record(json!({"a": 1}));
2015 let id = memory.write(&record, None).await.unwrap();
2016 assert_eq!(id, record.memory_id);
2017 // A retried submission of the same record converges.
2018 assert!(!store.put(&record).await.unwrap());
2019
2020 let events = journal.events();
2021 assert_eq!(events.len(), 1);
2022 let event = &events[0];
2023 assert_eq!(event.kind, RunEventKind::MemoryWrite);
2024 assert_eq!(event.effect, Effect::Idempotent);
2025 let input = event.input.as_ref().unwrap();
2026 let PayloadRef::Inline(input) = input else {
2027 panic!("write request travels inline")
2028 };
2029 assert_eq!(
2030 input["effect_key"].as_str().unwrap(),
2031 memory_effect_key(&record.scope, &record.memory_id)
2032 );
2033 }
2034
2035 #[tokio::test]
2036 async fn write_fails_during_replay() {
2037 let journal = Journal::new("run-1", "thread-1", Clock::System);
2038 let snapshot = journal.snapshot();
2039 let memory = journal.memory(MemorySource::Replay(MemoryReplaySource::new(&snapshot)));
2040 let err = memory
2041 .write(&record(json!({"a": 1})), None)
2042 .await
2043 .unwrap_err();
2044 assert!(matches!(err, RustyError::Replay(_)));
2045 }
2046}