Skip to main content

rusty_agent_runtime/
memory.rs

1//! Governed memory (R0.8 Rusty Learn, waves 1–2): the record model, the
2//! structured retrieval contract with its token-bounded deterministic
3//! assembly, the journaled read/write seam, and — from wave 2 — the
4//! correction loop and the memory operations (consolidation, conflict
5//! detection, forgetting) as journaled, evidence-carrying transitions.
6//!
7//! The design doc is `docs/learn-design.md` ("Governed memory"); the
8//! learning rule it answers to: **no learning process may silently rewrite a
9//! production prompt, graph, policy, memory, or tool permission.** Memory is
10//! therefore a governed record store, not a JSON blob an agent edits through
11//! a tool call:
12//!
13//! - [`MemoryRecord`] — one serde-versioned, content-addressed record:
14//!   identity is `sha256` over content plus provenance, so a changed record
15//!   is a new id and there is no in-place update anywhere in the model.
16//!   Supersession is a chain of immutable records; the superseded record is
17//!   retained as evidence but filtered from default retrieval.
18//! - [`MemoryScope`] — the five-scope taxonomy: `run` / `agent` / `team` /
19//!   `user` / `tenant`. It maps onto R0.7's four [`crate::agents::StateScope`]s
20//!   with one honest adaptation, stated exactly as the design draws it:
21//!   `StateScope` has no `run` member, because state outlives runs by
22//!   design; run scope is new here (memory whose lifetime is bound to one
23//!   run's thread); and **`agent` scope is `StateScope::Private` under its
24//!   memory name**. The taxonomy is a superset, not a fork: an agent
25//!   manifest's declared `StateScope`s translate one-to-one into the memory
26//!   scopes it may write ([`MemoryScope::from_state_scope`]).
27//! - [`MemoryProvenance`] — who wrote it (`agent:{id}` / `human:{id}` /
28//!   `distiller:{name}` / `system`), from what evidence (run id + journal
29//!   event ids, a correction id, a candidate id, the source record ids of a
30//!   consolidation), and when. Mandatory: a record that cannot name its
31//!   origin cannot be audited.
32//! - [`MemoryQuery`] + [`ContextBudget`] — deliberately structural
33//!   retrieval (scope, kind, key/tag equality, validity-at-time, minimum
34//!   confidence, exclude-expired, exclude-superseded, authored-by; **no
35//!   similarity search** in R0.8), packed into a prompt assembly by a
36//!   deterministic rank (explicit priority, then confidence, then recency,
37//!   with the content address as the final tie-break) until the token budget
38//!   is exhausted.
39//!
40//! # The journaled seam
41//!
42//! A memory read is an [`Effect::ReadOnly`] effect journaled as
43//! [`RunEventKind::MemoryRead`], with the resolved query plus budget as the
44//! input payload and the assembly itself — the record ids and their order,
45//! plus the token accounting — as the output payload. Two properties the
46//! design requires follow from this module's discipline:
47//!
48//! - **Determinism.** Equal store state and equal budget produce byte-equal
49//!   assemblies ([`assemble`] is a pure function with a total order), and
50//!   the resolved query — including the `as_of` timestamp, stamped through
51//!   the run's clock when unset — travels in the journaled request, so the
52//!   request hash fully determines the result.
53//! - **Replay-serving.** Exact replay serves the journaled assembly instead
54//!   of re-querying the store, per the rule the Flight Recorder already
55//!   applies to journaled model and tool calls. [`MemoryReplaySource`] is
56//!   the serving cursor (sequence + request hash, the same matching rule
57//!   [`crate::replay::ReplaySource`] applies to model/tool calls), and
58//!   [`JournaledMemory::read`] re-journals the served event byte-identically
59//!   — including clock-read parity with the live path, so a logical clock's
60//!   tick sequence stays aligned with the recorded run.
61//!
62//! A memory write is an [`Effect::Idempotent`] effect under the derived key
63//! `memory:{scope}:{memory_id}` ([`memory_effect_key`]): retried submissions
64//! converge, and the write is journaled as [`RunEventKind::MemoryWrite`]
65//! with its provenance. Writes are never served during replay — a replayed
66//! run issuing a write has diverged from its evidence.
67//!
68//! # The correction loop and the memory operations (wave 2)
69//!
70//! Wave 2 lands the correction loop's record-plane half: [`Correction`] is
71//! the highest-trust input the learning system has, so the loop's rule is
72//! **a correction becomes an attributed candidate memory or example — never
73//! an in-place rewrite of what it corrects**. The derived record carries
74//! `human:{author}` provenance with the correction id in evidence (the
75//! design's `human:{author} via correction:{id}` attribution), confidence
76//! 1.0, and — at agent scope or wider — a [`Candidacy`] mark: candidacy,
77//! not adoption, because a wrong human correction at tenant scope is a
78//! production incident with a name attached and evaluation is cheap
79//! insurance. Run scope adopts directly: it affects only the run that
80//! produced it. Same-key correction-sourced writes auto-supersede the prior
81//! record (open question 5's asymmetry: corrections are trusted because
82//! they are attributed; distillations are not because they are inferred).
83//!
84//! The three memory operations are journaled transitions over the store,
85//! never background daemons:
86//!
87//! - **Consolidation** — [`consolidation_summary`] builds the one `summary`
88//!   record that distills N sources: it names them in
89//!   [`MemoryEvidence::source_memory_ids`] (which is also what makes
90//!   dependent-summary invalidation computable on forgetting) and
91//!   supersedes them — [`apply_query`] treats a summary's sources as
92//!   superseded, so default retrieval serves the summary alone. The
93//!   distillation semantics stay with the caller; the runtime owns the
94//!   record's invariants.
95//! - **Conflict detection** — [`detect_conflicts`] flags live records that
96//!   share a key, overlap in validity, and carry contradictory content.
97//!   It flags ([`MemoryConflict`], a review item); it never resolves.
98//! - **Forgetting** — [`plan_forget`] computes the erasure before anything
99//!   is deleted: the targets plus the dependent summaries invalidated by
100//!   walking the source naming in reverse, transitively. Deletion is real
101//!   (derived state is erasable; run journals are hash-chained evidence and
102//!   are not — open question 4), and the receipt is the journaled
103//!   [`RunEventKind::MemoryForget`] tombstone: [`MemoryForgetTombstone`]
104//!   carries the id, scope, reason, and dependent invalidations — metadata
105//!   by construction, with no content field a careless serializer could
106//!   leak the forgotten bytes through.
107//!
108//! # Token accounting
109//!
110//! The budget is enforced in **estimated tokens: serialized content bytes ÷
111//! 4, plus a declared safety margin** ([`ContextBudget::margin_percent`],
112//! default [`DEFAULT_TOKEN_MARGIN_PERCENT`]), recorded as such on every
113//! assembly ([`TokenAccounting`]). This is the design's open-question-3
114//! default, stated plainly: provider-precise tokenizers differ per model and
115//! are heavyweight; model-precise counting plugs in later behind the same
116//! [`ContextBudget`] type, and until then the margin is the honest hedge.
117//!
118//! Golden-file tests under `tests/golden/` pin every wire shape in this
119//! module; any accidental contract drift fails CI.
120
121use std::collections::{BTreeMap, HashSet};
122use std::sync::{Arc, Mutex, MutexGuard};
123
124use async_trait::async_trait;
125use chrono::{DateTime, Utc};
126use serde::{Deserialize, Serialize};
127use serde_json::{json, Value};
128
129use crate::agents::StateScope;
130use crate::error::{Result, RustyError};
131use crate::journal::{EventDraft, Journal, JournalSnapshot};
132use crate::record::{
133    sha256_hex, ArtifactRef, Effect, PayloadRef, RunEvent, RunEventKind, INLINE_PAYLOAD_MAX_BYTES,
134};
135use crate::replay::ServedEffect;
136
137/// The memory schema version a run pins in its manifest
138/// ([`crate::record::RunManifest::memory_schema`]). Bump only on a breaking
139/// change to the record model; additive evolution uses serde defaults so
140/// previously written records keep deserializing.
141pub const MEMORY_SCHEMA_VERSION: &str = "memory-v1";
142
143/// The divisor of the estimated-token accounting: estimated tokens are
144/// serialized content bytes ÷ `TOKEN_BYTES_PER_ESTIMATE`, rounded up, before
145/// the safety margin. See the module docs for why the estimate — not a
146/// provider-precise tokenizer — is the R0.8 default.
147pub const TOKEN_BYTES_PER_ESTIMATE: u64 = 4;
148
149/// The default safety margin of the estimated-token accounting, in percent
150/// (design open question 3's "declared safety margin"): a record's estimated
151/// cost is `ceil(bytes / 4) * (1 + margin)`, so tokenizer drift against the
152/// estimate eats the margin before it eats the budget.
153pub const DEFAULT_TOKEN_MARGIN_PERCENT: u32 = 20;
154
155fn invalid(message: impl Into<String>) -> RustyError {
156    // A memory write is a state update to the governed store; contract
157    // validation failures reuse the invalid-update class rather than
158    // growing the error taxonomy for one module.
159    RustyError::InvalidUpdate(message.into())
160}
161
162fn replay_error(message: impl Into<String>) -> RustyError {
163    RustyError::Replay(message.into())
164}
165
166// --------------------------------------------------------------------- //
167// The record model
168// --------------------------------------------------------------------- //
169
170/// What a memory record is. Closed enum — retrieval and consolidation match
171/// exhaustively on it.
172#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
173#[serde(rename_all = "snake_case")]
174pub enum MemoryKind {
175    /// An asserted fact about the world the agent operates in.
176    Fact,
177    /// A stated preference (a user's, a team's), meant to shape future
178    /// behavior within its scope.
179    Preference,
180    /// A corrected input/output pair — the correction loop's output (R0.8
181    /// wave 2): the input a run saw and the behavior it should have
182    /// produced.
183    Example,
184    /// Consolidation's output: one record distilling N sources, which it
185    /// names in [`MemoryEvidence::source_memory_ids`] — the naming is what
186    /// makes dependent-summary invalidation computable on forgetting.
187    Summary,
188}
189
190/// The governance state of a correction-derived record (R0.8 wave 2).
191/// Closed enum with the single honest state this wave needs — wave 3's
192/// evaluation outcomes extend it additively, the same evolution rule
193/// [`RunEventKind`] follows. Not part of the content address: candidacy is
194/// governance state, not identity — the same reading `tags` and `priority`
195/// already take.
196#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
197#[serde(rename_all = "snake_case")]
198pub enum Candidacy {
199    /// Attributed and stored, but unevaluated. The design's scope-decides-
200    /// the-path rule: a correction at agent scope or wider becomes a
201    /// candidate, because a wrong human correction at tenant scope is a
202    /// production incident with a name attached, and the evaluation step is
203    /// cheap insurance against it.
204    Pending,
205}
206
207/// The scope a memory lives at: whose memory it is and how far its
208/// visibility reaches. Closed enum; the superset mapping onto R0.7's
209/// [`StateScope`] taxonomy is the module's load-bearing contract (see the
210/// module docs and [`MemoryScope::from_state_scope`]).
211#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
212#[serde(rename_all = "snake_case")]
213pub enum MemoryScope {
214    /// Memory whose lifetime is bound to one run's thread. New in R0.8:
215    /// `StateScope` has no `run` member, because state outlives runs by
216    /// design. Run scope is written by the runtime on the run's behalf.
217    Run,
218    /// One agent's own memory — `StateScope::Private` under its memory
219    /// name. Scope authorization is the manifest check: an agent may write
220    /// agent scope only when its `CapabilityManifest` declares
221    /// `StateScope::Private`.
222    Agent,
223    /// Memory shared by a team's members — `StateScope::Team` under its
224    /// memory name, under the same turn discipline that governs team state.
225    Team,
226    /// One end user's memory, shared across that user's agents and threads —
227    /// `StateScope::User` under its memory name.
228    User,
229    /// Configuration-grade memory for the whole tenant — `StateScope::Tenant`
230    /// under its memory name. Writable by operators, not by agents; tenant
231    /// isolation is the `{tenant}/` id-namespacing unchanged.
232    Tenant,
233}
234
235impl MemoryScope {
236    /// The memory scope a declared [`StateScope`] translates into, one-to-one
237    /// (`private` → `agent`): the manifest check's mapping. `None` is
238    /// unreachable — the four `StateScope`s all have memory names — but the
239    /// signature keeps the mapping honest about direction: this is the
240    /// superset reading the subset.
241    pub fn from_state_scope(scope: StateScope) -> Option<MemoryScope> {
242        Some(match scope {
243            StateScope::Private => MemoryScope::Agent,
244            StateScope::Team => MemoryScope::Team,
245            StateScope::User => MemoryScope::User,
246            StateScope::Tenant => MemoryScope::Tenant,
247        })
248    }
249
250    /// The [`StateScope`] this memory scope corresponds to, when it has one:
251    /// `agent` → `private`, `team` → `team`, `user` → `user`,
252    /// `tenant` → `tenant`, and `run` → `None` — run scope has no
253    /// `StateScope` member because state outlives runs by design.
254    pub fn to_state_scope(self) -> Option<StateScope> {
255        match self {
256            MemoryScope::Run => None,
257            MemoryScope::Agent => Some(StateScope::Private),
258            MemoryScope::Team => Some(StateScope::Team),
259            MemoryScope::User => Some(StateScope::User),
260            MemoryScope::Tenant => Some(StateScope::Tenant),
261        }
262    }
263}
264
265/// A [`MemoryScope`] plus the concrete scope id (agent id, user id, team id,
266/// run id, or the tenant name). Scope ids are tenant-relative names: the
267/// store namespaces every record under its tenant, so an id never needs to
268/// re-encode the tenant it belongs to.
269#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
270pub struct ScopeAddress {
271    /// The taxonomy member.
272    pub scope: MemoryScope,
273
274    /// The concrete id inside the scope (e.g. `researcher-7` for
275    /// `agent`-scoped memory).
276    pub id: String,
277}
278
279impl ScopeAddress {
280    /// Build an address; validation of the id happens at the write gate,
281    /// not here (addresses also arrive from stored evidence).
282    pub fn new(scope: MemoryScope, id: impl Into<String>) -> Self {
283        Self {
284            scope,
285            id: id.into(),
286        }
287    }
288
289    /// The canonical `"{scope}:{id}"` string form, used in derived effect
290    /// keys (`memory:{scope}:{memory_id}`) and human-facing provenance.
291    pub fn as_address(&self) -> String {
292        let scope = serde_json::to_value(self.scope)
293            .ok()
294            .and_then(|v| v.as_str().map(str::to_owned))
295            .unwrap_or_else(|| format!("{:?}", self.scope).to_lowercase());
296        format!("{scope}:{}", self.id)
297    }
298}
299
300/// Who wrote a memory record. Closed enum: a record that cannot name its
301/// origin cannot be audited, so provenance is mandatory, not optional — and
302/// the attribution travels with the record through every later consumer
303/// (distiller, evaluator, auditor).
304#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
305#[serde(tag = "type", rename_all = "snake_case")]
306pub enum ProvenanceAuthor {
307    /// An agent wrote it (in a governed write inside a run): `agent:{id}`.
308    Agent {
309        /// The writing agent's id (tenant-relative).
310        agent_id: String,
311    },
312    /// A human wrote it — a correction, the highest-trust input the learning
313    /// system has: `human:{id}`. Human-authored records default to
314    /// confidence 1.0 (the claim is the person's, stated plainly).
315    Human {
316        /// The correcting human's identity. Mandatory because a correction
317        /// that cannot name its corrector is indistinguishable from a
318        /// prompt edit.
319        human_id: String,
320    },
321    /// A distiller produced it from recorded evidence: `distiller:{name}`.
322    Distiller {
323        /// The distiller's name (application code, not runtime code).
324        name: String,
325    },
326    /// The runtime itself wrote it (e.g. a run-scope record written on the
327    /// run's behalf): `system`.
328    System,
329}
330
331impl ProvenanceAuthor {
332    /// The canonical id string (`agent:{id}` / `human:{id}` /
333    /// `distiller:{name}` / `system`), as the design's provenance
334    /// vocabulary spells it.
335    pub fn as_id_string(&self) -> String {
336        match self {
337            ProvenanceAuthor::Agent { agent_id } => format!("agent:{agent_id}"),
338            ProvenanceAuthor::Human { human_id } => format!("human:{human_id}"),
339            ProvenanceAuthor::Distiller { name } => format!("distiller:{name}"),
340            ProvenanceAuthor::System => "system".to_owned(),
341        }
342    }
343}
344
345/// What a memory record was derived from: the evidence an auditor walks.
346/// Every field is optional and absent from the wire when unset — a
347/// human-authored fact may carry no derivation at all, and sparse evidence
348/// must not change the shape for dense readers.
349#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
350pub struct MemoryEvidence {
351    /// The run whose journal the record was distilled or corrected from.
352    #[serde(default, skip_serializing_if = "Option::is_none")]
353    pub run_id: Option<String>,
354
355    /// The journal event ids inside `run_id` the record draws on.
356    #[serde(default, skip_serializing_if = "Vec::is_empty")]
357    pub event_ids: Vec<String>,
358
359    /// The correction (R0.8 wave 2) the record was derived from.
360    #[serde(default, skip_serializing_if = "Option::is_none")]
361    pub correction_id: Option<String>,
362
363    /// The candidate (R0.8 wave 3) whose promotion carried this record.
364    #[serde(default, skip_serializing_if = "Option::is_none")]
365    pub candidate_id: Option<String>,
366
367    /// The records a `summary` consolidated — the naming that makes
368    /// dependent-summary invalidation computable on forgetting.
369    #[serde(default, skip_serializing_if = "Vec::is_empty")]
370    pub source_memory_ids: Vec<String>,
371}
372
373impl MemoryEvidence {
374    /// `true` when no evidence is carried (the sparse wire shape).
375    pub fn is_empty(&self) -> bool {
376        self.run_id.is_none()
377            && self.event_ids.is_empty()
378            && self.correction_id.is_none()
379            && self.candidate_id.is_none()
380            && self.source_memory_ids.is_empty()
381    }
382}
383
384/// A record's origin: who wrote it, from what evidence, and when. Part of
385/// the content address ([`MemoryRecord::new`]) — two records with identical
386/// content but different origins are different records, because they answer
387/// differently to an audit.
388#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
389pub struct MemoryProvenance {
390    /// Who wrote it.
391    pub author: ProvenanceAuthor,
392
393    /// What it was derived from. Absent from the wire when empty.
394    #[serde(default, skip_serializing_if = "MemoryEvidence::is_empty")]
395    pub evidence: MemoryEvidence,
396
397    /// When the system learned it. Distinct from
398    /// [`ValidityWindow::valid_from`] (when the record claims to be true) —
399    /// the bitemporal split, kept as two plain timestamps (Zep's validity
400    /// window as a flat field).
401    pub written_at: DateTime<Utc>,
402}
403
404/// The interval the record claims to be true: `[valid_from, valid_until)`,
405/// open-ended when `valid_until` is `None`. Contradiction is handled by
406/// time (and supersession), never by deletion.
407#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
408pub struct ValidityWindow {
409    /// Inclusive start of the claimed-true interval.
410    pub valid_from: DateTime<Utc>,
411
412    /// Exclusive end of the claimed-true interval; `None` = open-ended.
413    #[serde(default, skip_serializing_if = "Option::is_none")]
414    pub valid_until: Option<DateTime<Utc>>,
415}
416
417impl ValidityWindow {
418    /// A window starting at `valid_from`, open-ended.
419    pub fn starting(valid_from: DateTime<Utc>) -> Self {
420        Self {
421            valid_from,
422            valid_until: None,
423        }
424    }
425
426    /// Whether the window claims to be true at `t`: inclusive start,
427    /// exclusive end.
428    pub fn contains(&self, t: DateTime<Utc>) -> bool {
429        self.valid_from <= t && self.valid_until.is_none_or(|until| t < until)
430    }
431}
432
433fn is_false(value: &bool) -> bool {
434    !*value
435}
436
437fn is_zero(value: &i64) -> bool {
438    *value == 0
439}
440
441/// One governed memory record: content-addressed, immutable, scoped,
442/// attributed.
443///
444/// Every field exists because a downstream operation needs it (the design's
445/// rule): retrieval filters on scope/kind/key/tags/confidence/validity/
446/// expiration/supersession/author; the assembly ranks on
447/// priority/confidence/recency; the audit walks provenance; forgetting walks
448/// `supersedes` in reverse. The one reserved field is `embedding` —
449/// reserved so vector retrieval slots in additively when the roadmap's
450/// de-prioritization lifts; nothing in R0.8 reads it.
451#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
452pub struct MemoryRecord {
453    /// The content address: `sha256` over the canonical serialization of the
454    /// record's content plus provenance (the one hashing primitive shared
455    /// with artifact references and journal heads). Immutable by
456    /// construction: a changed record is a new id.
457    pub memory_id: String,
458
459    /// What the record is.
460    pub kind: MemoryKind,
461
462    /// Whose memory it is.
463    pub scope: ScopeAddress,
464
465    /// The writer-declared lookup key, when the record answers a named
466    /// question ("what is user-7's timezone"). With no vector retrieval in
467    /// R0.8, keying is deliberate: absence of a hit is absence of a key, not
468    /// absence of a fact.
469    #[serde(default, skip_serializing_if = "Option::is_none")]
470    pub key: Option<String>,
471
472    /// Writer-declared tags; retrieval matches by equality (a record must
473    /// carry every queried tag).
474    #[serde(default, skip_serializing_if = "Vec::is_empty")]
475    pub tags: Vec<String>,
476
477    /// The writer-declared explicit priority — the assembly rank's first
478    /// input (higher packs first). `0` is the default and stays absent from
479    /// the wire, so pre-priority records keep their shape.
480    #[serde(default, skip_serializing_if = "is_zero")]
481    pub priority: i64,
482
483    /// Who wrote it, from what evidence, and when. Mandatory.
484    pub provenance: MemoryProvenance,
485
486    /// The writer-declared confidence in `(0, 1]`. Retrieval filters on it;
487    /// nothing in the runtime *computes* it — confidence is a claim, not a
488    /// measurement, and the model is honest about that.
489    pub confidence: f64,
490
491    /// The interval the record claims to be true.
492    pub validity: ValidityWindow,
493
494    /// When the system learned it (the bitemporal split's other timestamp;
495    /// duplicates [`MemoryProvenance::written_at`] deliberately — the record
496    /// is self-contained when provenance is summarized away by a consumer).
497    pub created_at: DateTime<Utc>,
498
499    /// Optional TTL. Expiration is a retrieval filter (and, from wave 2, a
500    /// forgetting trigger), not a silent reaper.
501    #[serde(default, skip_serializing_if = "Option::is_none")]
502    pub expires_at: Option<DateTime<Utc>>,
503
504    /// The record this one replaces, when it does. Supersession is a
505    /// chain of immutable records; the superseded record is retained as
506    /// evidence but filtered from default retrieval.
507    #[serde(default, skip_serializing_if = "Option::is_none")]
508    pub supersedes: Option<String>,
509
510    /// Set when the record is a correction-derived candidate pending
511    /// evaluation (R0.8 wave 2; [`Candidacy`]). Additive: absent from the
512    /// wire while unset, and outside the content address (candidacy is
513    /// governance state, not identity).
514    #[serde(default, skip_serializing_if = "Option::is_none")]
515    pub candidacy: Option<Candidacy>,
516
517    /// The record body: inline at or below [`INLINE_PAYLOAD_MAX_BYTES`],
518    /// content-addressed above — the journal's own payload discipline, so
519    /// memory bodies share artifact storage and the large-body story needs
520    /// nothing new.
521    pub content: PayloadRef,
522
523    /// Reserved for vector retrieval (deferred, per the design's not-built
524    /// list). Additive by construction: absent from the wire while unset,
525    /// so the field lands without a wire change when it earns its keep.
526    #[serde(default, skip_serializing_if = "Option::is_none")]
527    pub embedding: Option<Value>,
528}
529
530impl MemoryRecord {
531    /// Build a record, deriving its content address from the content plus
532    /// provenance and splitting the content per the payload discipline
533    /// (inline ≤ [`INLINE_PAYLOAD_MAX_BYTES`], content-addressed above).
534    ///
535    /// Fails when `confidence` is outside `(0, 1]` — the one validity rule
536    /// the contract enforces at construction, because a confidence outside
537    /// the interval is not a claim at all.
538    pub fn new(
539        kind: MemoryKind,
540        scope: ScopeAddress,
541        provenance: MemoryProvenance,
542        confidence: f64,
543        validity: ValidityWindow,
544        created_at: DateTime<Utc>,
545        content: Value,
546    ) -> Result<Self> {
547        if !(confidence > 0.0 && confidence <= 1.0) {
548            return Err(invalid(format!(
549                "memory confidence must be in (0, 1], got {confidence} — confidence is a \
550                 writer-declared claim, and a value outside the interval is not a claim at all"
551            )));
552        }
553        let content_ref = content_ref_of(&content);
554        let memory_id = derive_memory_id(&content_ref, &provenance)?;
555        Ok(Self {
556            memory_id,
557            kind,
558            scope,
559            key: None,
560            tags: Vec::new(),
561            priority: 0,
562            provenance,
563            confidence,
564            validity,
565            created_at,
566            expires_at: None,
567            supersedes: None,
568            candidacy: None,
569            content: content_ref,
570            embedding: None,
571        })
572    }
573
574    /// Builder-style: set the lookup key.
575    pub fn with_key(mut self, key: impl Into<String>) -> Self {
576        self.key = Some(key.into());
577        self
578    }
579
580    /// Builder-style: set the tags.
581    pub fn with_tags(mut self, tags: impl IntoIterator<Item = impl Into<String>>) -> Self {
582        self.tags = tags.into_iter().map(Into::into).collect();
583        self
584    }
585
586    /// Builder-style: set the explicit priority (the assembly rank's first
587    /// input).
588    pub fn with_priority(mut self, priority: i64) -> Self {
589        self.priority = priority;
590        self
591    }
592
593    /// Builder-style: set the TTL.
594    pub fn with_expires_at(mut self, expires_at: DateTime<Utc>) -> Self {
595        self.expires_at = Some(expires_at);
596        self
597    }
598
599    /// Builder-style: name the record this one supersedes.
600    pub fn with_supersedes(mut self, supersedes: impl Into<String>) -> Self {
601        self.supersedes = Some(supersedes.into());
602        self
603    }
604
605    /// Builder-style: mark the record as a correction-derived candidate
606    /// (R0.8 wave 2; [`Candidacy`]).
607    pub fn with_candidacy(mut self, candidacy: Candidacy) -> Self {
608        self.candidacy = Some(candidacy);
609        self
610    }
611
612    /// The serialized size of the content in bytes — the input to the
613    /// estimated-token accounting. Inline payloads serialize on demand;
614    /// artifact-referenced payloads carry their size on the reference (the
615    /// canonical-serialization size the address was minted from).
616    pub fn content_bytes(&self) -> u64 {
617        match &self.content {
618            PayloadRef::Inline(value) => serde_json::to_vec(value)
619                .map(|bytes| bytes.len() as u64)
620                .unwrap_or(0),
621            PayloadRef::Artifact(reference) => reference.bytes,
622        }
623    }
624
625    /// Split `content` per the payload discipline, as [`MemoryRecord::new`]
626    /// does: inline at or below [`INLINE_PAYLOAD_MAX_BYTES`],
627    /// content-addressed above. The artifact bytes are the canonical
628    /// serialization of `content`; the caller persists them (the artifact
629    /// store shared with the journal) under the reference's address.
630    pub fn content_ref_of(content: &Value) -> PayloadRef {
631        content_ref_of(content)
632    }
633}
634
635/// The payload-discipline split, free-standing so both the record
636/// constructor and store backends share exactly one decision.
637fn content_ref_of(content: &Value) -> PayloadRef {
638    let Ok(bytes) = serde_json::to_vec(content) else {
639        // A Value that fails to serialize is a contradiction; keep it
640        // inline rather than drop evidence (the journal's own rule).
641        return PayloadRef::Inline(content.clone());
642    };
643    if bytes.len() <= INLINE_PAYLOAD_MAX_BYTES {
644        return PayloadRef::Inline(content.clone());
645    }
646    PayloadRef::Artifact(ArtifactRef {
647        sha256: sha256_hex(&bytes),
648        bytes: bytes.len() as u64,
649    })
650}
651
652/// The content address of a record: `sha256` over the canonical
653/// serialization of `{content, provenance}` — content by hash (so inline
654/// and artifact-referenced forms of the same body address identically) and
655/// provenance in full (origin is identity).
656pub fn derive_memory_id(content: &PayloadRef, provenance: &MemoryProvenance) -> Result<String> {
657    let content_hash = content.content_hash()?;
658    let identity = json!({ "content": content_hash, "provenance": provenance });
659    Ok(sha256_hex(&serde_json::to_vec(&identity)?))
660}
661
662// --------------------------------------------------------------------- //
663// Retrieval: structured filters + context budget
664// --------------------------------------------------------------------- //
665
666/// A structured memory query. Deliberately **not** semantic: R0.8 has no
667/// similarity search (the design's not-built list), so writers key and tag
668/// records deliberately and consumers treat absence of a hit as absence of
669/// a key, not absence of a fact.
670///
671/// Every field is optional; an empty query matches everything in the
672/// namespace (subject to the two defaults: expired and superseded records
673/// are excluded unless explicitly included). The query is journaled as the
674/// read's request, so its serialization is part of the replay contract —
675/// sparse fields stay absent, and [`MemoryQuery::as_of`] is always resolved
676/// before journaling (see [`JournaledMemory::read`]).
677#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
678pub struct MemoryQuery {
679    /// Restrict to one scope address (whose memory, concretely).
680    #[serde(default, skip_serializing_if = "Option::is_none")]
681    pub scope: Option<ScopeAddress>,
682
683    /// Restrict to these kinds (empty = all kinds).
684    #[serde(default, skip_serializing_if = "Vec::is_empty")]
685    pub kinds: Vec<MemoryKind>,
686
687    /// Key equality.
688    #[serde(default, skip_serializing_if = "Option::is_none")]
689    pub key: Option<String>,
690
691    /// Tag equality: a record must carry every listed tag.
692    #[serde(default, skip_serializing_if = "Vec::is_empty")]
693    pub tags: Vec<String>,
694
695    /// Validity-at-time: the record's validity window must contain this
696    /// instant (the bitemporal "true as of" filter).
697    #[serde(default, skip_serializing_if = "Option::is_none")]
698    pub valid_at: Option<DateTime<Utc>>,
699
700    /// Minimum writer-declared confidence, inclusive.
701    #[serde(default, skip_serializing_if = "Option::is_none")]
702    pub min_confidence: Option<f64>,
703
704    /// Include records past their `expires_at` (default: excluded).
705    /// Expiration is a filter, not a reaper — the records are still there.
706    #[serde(default, skip_serializing_if = "is_false")]
707    pub include_expired: bool,
708
709    /// Include records another record supersedes (default: excluded — the
710    /// superseded record is retained as evidence, filtered from default
711    /// retrieval).
712    #[serde(default, skip_serializing_if = "is_false")]
713    pub include_superseded: bool,
714
715    /// Restrict to one author.
716    #[serde(default, skip_serializing_if = "Option::is_none")]
717    pub authored_by: Option<ProvenanceAuthor>,
718
719    /// Restrict to correction-derived candidates pending evaluation (R0.8
720    /// wave 2; default: all records, candidates or not). This is how the
721    /// evaluation half of the loop finds its queue: candidacy is a
722    /// governance mark, so it filters like one — structurally, never by
723    /// content.
724    #[serde(default, skip_serializing_if = "is_false")]
725    pub candidates_only: bool,
726
727    /// The instant expiry is evaluated against. When unset, the reader
728    /// stamps it through the run's clock at read time (wall clock for live
729    /// reads; the deterministic logical clock for recorded runs) — so the
730    /// journaled request always carries the resolved value and a replayed
731    /// read re-derives the same one.
732    #[serde(default, skip_serializing_if = "Option::is_none")]
733    pub as_of: Option<DateTime<Utc>>,
734}
735
736impl MemoryQuery {
737    /// Whether `record` matches every declared filter. `superseded` tells
738    /// whether another record in the queried universe supersedes this one
739    /// (computable only against the whole set, so the caller supplies it);
740    /// `now` is the resolved expiry instant ([`MemoryQuery::as_of`]).
741    pub fn matches(&self, record: &MemoryRecord, superseded: bool, now: DateTime<Utc>) -> bool {
742        if let Some(scope) = &self.scope {
743            if &record.scope != scope {
744                return false;
745            }
746        }
747        if !self.kinds.is_empty() && !self.kinds.contains(&record.kind) {
748            return false;
749        }
750        if let Some(key) = &self.key {
751            if record.key.as_deref() != Some(key.as_str()) {
752                return false;
753            }
754        }
755        if !self
756            .tags
757            .iter()
758            .all(|tag| record.tags.iter().any(|t| t == tag))
759        {
760            return false;
761        }
762        if let Some(valid_at) = self.valid_at {
763            if !record.validity.contains(valid_at) {
764                return false;
765            }
766        }
767        if let Some(min_confidence) = self.min_confidence {
768            if record.confidence < min_confidence {
769                return false;
770            }
771        }
772        if !self.include_expired && record.expires_at.is_some_and(|expires| expires <= now) {
773            return false;
774        }
775        if !self.include_superseded && superseded {
776            return false;
777        }
778        if let Some(author) = &self.authored_by {
779            if &record.provenance.author != author {
780                return false;
781            }
782        }
783        if self.candidates_only && record.candidacy.is_none() {
784            return false;
785        }
786        true
787    }
788}
789
790/// Apply `query` to a universe of records (one tenant's namespace), pure:
791/// the superseded set is computed over the universe, then every declared
792/// filter runs through [`MemoryQuery::matches`]. Both store backends share
793/// this function — the JSON backend scans and applies it directly; Postgres
794/// pre-filters on columns and applies the same function to the reduced set,
795/// so filter semantics live in exactly one place.
796///
797/// The superseded set has two halves. A record is superseded when another
798/// record names it in `supersedes` (the wave-1 rule) — **or, from wave 2,
799/// when a `summary` record names it in
800/// [`MemoryEvidence::source_memory_ids`]**: consolidation supersedes the
801/// records it distills, and the naming is the same one dependent-summary
802/// invalidation walks on forgetting.
803pub fn apply_query(
804    universe: &[MemoryRecord],
805    query: &MemoryQuery,
806    now: DateTime<Utc>,
807) -> Vec<MemoryRecord> {
808    let superseded: HashSet<&str> = superseded_set(universe);
809    universe
810        .iter()
811        .filter(|record| query.matches(record, superseded.contains(record.memory_id.as_str()), now))
812        .cloned()
813        .collect()
814}
815
816/// The tenant namespace's superseded set, as [`apply_query`] defines it:
817/// everything named in a `supersedes` field, plus everything a `summary`
818/// record names as a source (see its docs). Shared by conflict detection
819/// and forget planning, which must agree with retrieval about what is
820/// superseded — three readers, one definition.
821pub fn superseded_set(universe: &[MemoryRecord]) -> HashSet<&str> {
822    universe
823        .iter()
824        .filter_map(|record| record.supersedes.as_deref())
825        .chain(
826            universe
827                .iter()
828                .filter(|record| record.kind == MemoryKind::Summary)
829                .flat_map(|record| {
830                    record
831                        .provenance
832                        .evidence
833                        .source_memory_ids
834                        .iter()
835                        .map(String::as_str)
836                }),
837        )
838        .collect()
839}
840
841/// What the assembly does when the next-ranked record does not fit the
842/// remaining budget.
843#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
844#[serde(rename_all = "snake_case")]
845pub enum BudgetOverflow {
846    /// Stop packing: the assembly carries what fit, ranked, with
847    /// `truncated: true`. The default — a prompt assembly that silently
848    /// drops lower-ranked context is the honest behavior for a budget.
849    #[default]
850    Truncate,
851    /// Fail the read: the caller declared the budget hard and wants to
852    /// know the filtered set did not fit, rather than see a prefix of it.
853    Fail,
854}
855
856/// A prompt-assembly budget, in estimated tokens (see the module docs for
857/// the accounting rule). `ContextBudget` is the seam a model-precise
858/// tokenizer plugs into later: the type stays, the estimate behind
859/// [`estimated_tokens`] is what a later wave may sharpen.
860#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
861pub struct ContextBudget {
862    /// The maximum estimated tokens the assembly may carry.
863    pub max_tokens: u32,
864
865    /// The declared safety margin of the estimate, in percent
866    /// (default [`DEFAULT_TOKEN_MARGIN_PERCENT`]; absent from the wire at
867    /// the default).
868    #[serde(
869        default = "default_margin_percent",
870        skip_serializing_if = "is_default_margin_percent"
871    )]
872    pub margin_percent: u32,
873
874    /// What to do when the next-ranked record does not fit (default
875    /// [`BudgetOverflow::Truncate`]).
876    #[serde(default)]
877    pub overflow: BudgetOverflow,
878}
879
880fn default_margin_percent() -> u32 {
881    DEFAULT_TOKEN_MARGIN_PERCENT
882}
883
884fn is_default_margin_percent(value: &u32) -> bool {
885    *value == DEFAULT_TOKEN_MARGIN_PERCENT
886}
887
888impl ContextBudget {
889    /// A budget of `max_tokens` estimated tokens with the declared default
890    /// margin and truncate overflow.
891    pub fn new(max_tokens: u32) -> Self {
892        Self {
893            max_tokens,
894            margin_percent: DEFAULT_TOKEN_MARGIN_PERCENT,
895            overflow: BudgetOverflow::Truncate,
896        }
897    }
898
899    /// Builder-style: override the safety margin.
900    pub fn with_margin_percent(mut self, margin_percent: u32) -> Self {
901        self.margin_percent = margin_percent;
902        self
903    }
904
905    /// Builder-style: override the overflow policy.
906    pub fn with_overflow(mut self, overflow: BudgetOverflow) -> Self {
907        self.overflow = overflow;
908        self
909    }
910}
911
912/// Estimated tokens for `bytes` of serialized content under the declared
913/// accounting: `ceil(bytes / TOKEN_BYTES_PER_ESTIMATE)` scaled up by the
914/// margin, saturated at `u32::MAX` (an over-budget estimate only means the
915/// record does not fit — the estimate never wraps).
916pub fn estimated_tokens(bytes: u64, margin_percent: u32) -> u32 {
917    let base = bytes.div_ceil(TOKEN_BYTES_PER_ESTIMATE) as u128;
918    let scaled = base * (100 + margin_percent as u128) / 100;
919    scaled.min(u32::MAX as u128) as u32
920}
921
922/// The accounting an assembly was packed under, journaled with it: the
923/// estimate rule (bytes per token and margin), the budget, and how much of
924/// it the assembly used. Recorded — not recomputed — so an auditor reads
925/// the accounting the read actually applied.
926#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
927pub struct TokenAccounting {
928    /// The estimate's divisor ([`TOKEN_BYTES_PER_ESTIMATE`]): estimated
929    /// tokens are serialized content bytes divided by this.
930    pub bytes_per_token: u64,
931
932    /// The safety margin that was applied, in percent.
933    pub margin_percent: u32,
934
935    /// The budget the assembly was packed against, in estimated tokens.
936    pub budget_tokens: u32,
937
938    /// The estimated tokens the packed records sum to.
939    pub used_tokens: u32,
940}
941
942/// The result of a token-bounded retrieval: the filtered records, ranked
943/// and packed. This is the [`RunEventKind::MemoryRead`] event's output
944/// payload — the record ids and their order are first-class
945/// ([`MemoryAssembly::memory_ids`], the design's journaled assembly), and
946/// the full records travel with them so the prompt a model saw is
947/// reconstructable from the journal alone, never from a store query re-run
948/// later against mutated state.
949#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
950pub struct MemoryAssembly {
951    /// The packed records' content addresses, in rank order. The journaled
952    /// fact replay and audit match on.
953    pub memory_ids: Vec<String>,
954
955    /// The packed records, in the same order.
956    pub records: Vec<MemoryRecord>,
957
958    /// The accounting the packing applied.
959    pub token_accounting: TokenAccounting,
960
961    /// `true` when the budget cut the ranked set short (truncate overflow).
962    pub truncated: bool,
963}
964
965/// Rank and pack `records` under `budget`, deterministically.
966///
967/// The rank is a total order — explicit priority descending, then
968/// confidence descending, then recency (`created_at`) descending, then the
969/// content address ascending as the final tie-break — so equal store state
970/// and equal budget produce byte-equal assemblies. Packing walks the ranked
971/// list and stops at the first record that does not fit the remaining
972/// budget ([`BudgetOverflow::Truncate`]) or fails the read
973/// ([`BudgetOverflow::Fail`]).
974pub fn assemble(records: Vec<MemoryRecord>, budget: &ContextBudget) -> Result<MemoryAssembly> {
975    let mut ranked = records;
976    ranked.sort_by(|a, b| {
977        b.priority
978            .cmp(&a.priority)
979            .then_with(|| b.confidence.total_cmp(&a.confidence))
980            .then_with(|| b.created_at.cmp(&a.created_at))
981            .then_with(|| a.memory_id.cmp(&b.memory_id))
982    });
983    let mut used_tokens: u32 = 0;
984    let mut packed: Vec<MemoryRecord> = Vec::new();
985    let mut truncated = false;
986    for record in ranked {
987        let cost = estimated_tokens(record.content_bytes(), budget.margin_percent);
988        if used_tokens.saturating_add(cost) > budget.max_tokens {
989            match budget.overflow {
990                BudgetOverflow::Truncate => {
991                    truncated = true;
992                    break;
993                }
994                BudgetOverflow::Fail => {
995                    return Err(invalid(format!(
996                        "memory assembly exceeds the context budget: record `{}` costs an \
997                         estimated {cost} tokens with {} of {} already used — the filtered \
998                         set does not fit, and the budget was declared hard",
999                        record.memory_id, used_tokens, budget.max_tokens
1000                    )));
1001                }
1002            }
1003        }
1004        used_tokens = used_tokens.saturating_add(cost);
1005        packed.push(record);
1006    }
1007    let memory_ids = packed
1008        .iter()
1009        .map(|record| record.memory_id.clone())
1010        .collect();
1011    Ok(MemoryAssembly {
1012        memory_ids,
1013        records: packed,
1014        token_accounting: TokenAccounting {
1015            bytes_per_token: TOKEN_BYTES_PER_ESTIMATE,
1016            margin_percent: budget.margin_percent,
1017            budget_tokens: budget.max_tokens,
1018            used_tokens,
1019        },
1020        truncated,
1021    })
1022}
1023
1024// --------------------------------------------------------------------- //
1025// The correction loop (R0.8 wave 2)
1026// --------------------------------------------------------------------- //
1027
1028/// Reject an empty (or all-whitespace) identity at deserialization: a
1029/// correction that cannot name its corrector is indistinguishable from a
1030/// prompt edit, so attribution is enforced where the contract is parsed,
1031/// not where it is consumed.
1032fn deserialize_non_empty_id<'de, D>(deserializer: D) -> std::result::Result<String, D::Error>
1033where
1034    D: serde::Deserializer<'de>,
1035{
1036    let value = String::deserialize(deserializer)?;
1037    if value.trim().is_empty() {
1038        return Err(serde::de::Error::custom(
1039            "identity must be non-empty — attribution is mandatory",
1040        ));
1041    }
1042    Ok(value)
1043}
1044
1045/// What a correction corrects. Closed enum — the three targets the design
1046/// names; consumers match exhaustively.
1047#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
1048#[serde(tag = "type", rename_all = "snake_case")]
1049pub enum CorrectionTarget {
1050    /// A journaled run event (`{run_id}:{seq}`). The correction also yields
1051    /// an `example`-kind record — the input the run saw (read from the
1052    /// journaled event, never re-asked of the world) plus the corrected
1053    /// behavior — which the distiller folds into a versioned dataset: the
1054    /// correction is both the fix and the regression test.
1055    RunEvent {
1056        /// The run whose journal holds the event.
1057        run_id: String,
1058        /// The journaled event's id inside that run.
1059        event_id: String,
1060    },
1061    /// A memory record, by content address. The derived record inherits the
1062    /// target's key, so a same-key correction auto-supersedes the prior
1063    /// record (open question 5).
1064    Memory {
1065        /// The corrected record's content address.
1066        memory_id: String,
1067    },
1068    /// A pinned prompt hash from the run manifest
1069    /// ([`crate::record::RunManifest::pin_prompt`]).
1070    Prompt {
1071        /// The pinned prompt's content hash.
1072        prompt_hash: String,
1073    },
1074}
1075
1076/// A human correction: the highest-trust input the learning system has, and
1077/// the loop treats it accordingly — **a correction becomes an attributed
1078/// candidate memory or example, never an in-place rewrite of what it
1079/// corrects.**
1080///
1081/// `author` is mandatory and validated at deserialization (see
1082/// `deserialize_non_empty_id`); attribution then travels with every
1083/// derived record as `human:{author}` provenance with the correction id in
1084/// evidence — the design's `human:{author} via correction:{id}` string —
1085/// so every later consumer (distiller, evaluator, auditor) can trace the
1086/// record to the person and the moment.
1087#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
1088pub struct Correction {
1089    /// The correction's own id, caller-minted; carried in the derived
1090    /// records' evidence (and therefore their content addresses — two
1091    /// corrections with identical corrected content stay distinct records).
1092    #[serde(deserialize_with = "deserialize_non_empty_id")]
1093    pub correction_id: String,
1094
1095    /// The correcting human's identity. Mandatory.
1096    #[serde(deserialize_with = "deserialize_non_empty_id")]
1097    pub author: String,
1098
1099    /// What it corrects.
1100    pub target: CorrectionTarget,
1101
1102    /// The corrected content: what the derived record asserts.
1103    pub corrected: Value,
1104
1105    /// The scope the result should live at. Scope decides the path
1106    /// ([`Correction::is_candidate`]): run scope is adopted directly (it
1107    /// affects only the run that produced it); agent scope or wider becomes
1108    /// a candidate pending evaluation.
1109    pub scope: ScopeAddress,
1110
1111    /// Why the correction is right, when the corrector said. Optional; the
1112    /// attribution is not.
1113    #[serde(default, skip_serializing_if = "Option::is_none")]
1114    pub rationale: Option<String>,
1115}
1116
1117impl Correction {
1118    /// The derived records' author: `human:{author}`.
1119    pub fn author_as_provenance(&self) -> ProvenanceAuthor {
1120        ProvenanceAuthor::Human {
1121            human_id: self.author.clone(),
1122        }
1123    }
1124
1125    /// The attribution string the design spells: `human:{author} via
1126    /// correction:{id}`. The derived record carries it structurally (author
1127    /// plus evidence); this is the human-facing form, for responses and
1128    /// audit output.
1129    pub fn attribution(&self) -> String {
1130        format!(
1131            "human:{} via correction:{}",
1132            self.author, self.correction_id
1133        )
1134    }
1135
1136    /// The evidence every derived record carries: the correction id, plus
1137    /// the run linkage when the target is a journaled run event.
1138    pub fn evidence(&self) -> MemoryEvidence {
1139        let (run_id, event_ids) = match &self.target {
1140            CorrectionTarget::RunEvent { run_id, event_id } => {
1141                (Some(run_id.clone()), vec![event_id.clone()])
1142            }
1143            _ => (None, Vec::new()),
1144        };
1145        MemoryEvidence {
1146            run_id,
1147            event_ids,
1148            correction_id: Some(self.correction_id.clone()),
1149            ..MemoryEvidence::default()
1150        }
1151    }
1152
1153    /// The scope-decides-the-path rule: `true` when this correction's
1154    /// derived records are candidates pending evaluation (agent scope or
1155    /// wider), `false` at run scope, where a correction is adopted directly
1156    /// — it affects only the run that produced it.
1157    pub fn is_candidate(&self) -> bool {
1158        self.scope.scope != MemoryScope::Run
1159    }
1160}
1161
1162// --------------------------------------------------------------------- //
1163// Forgetting (R0.8 wave 2): real deletion with a journaled receipt
1164// --------------------------------------------------------------------- //
1165
1166/// Why a record was forgotten. Closed enum, carried on the tombstone: the
1167/// reason is what distinguishes a reaper's expiry from a GDPR-shaped
1168/// erasure request in the audit trail.
1169#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
1170#[serde(rename_all = "snake_case")]
1171pub enum ForgetReason {
1172    /// The record's TTL lapsed. Expiration stays a retrieval filter too —
1173    /// this is the operator-driven sweep acting on it, journaled.
1174    Expired,
1175    /// The record was wrong (or its correction superseded the need for it)
1176    /// and is withdrawn.
1177    Retracted,
1178    /// An erasure request (a user's, an operator's compliance pass).
1179    ErasureRequest,
1180}
1181
1182/// The journaled receipt of a forgetting — the
1183/// [`RunEventKind::MemoryForget`] event's output payload. **Metadata by
1184/// construction**: the forgotten id, its scope, the reason, and the
1185/// dependent invalidations. There is no content field to leave unset — a
1186/// tombstone that *could* carry the forgotten bytes would be one careless
1187/// serializer away from defeating the erasure it receipts.
1188#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
1189pub struct MemoryForgetTombstone {
1190    /// The forgotten record's content address.
1191    pub memory_id: String,
1192
1193    /// The scope the record lived at.
1194    pub scope: ScopeAddress,
1195
1196    /// Why it was forgotten.
1197    pub reason: ForgetReason,
1198
1199    /// The dependent summaries the erasure invalidated (the reverse walk
1200    /// over source naming, transitively — see [`plan_forget`]). Their ids,
1201    /// never their content.
1202    #[serde(default, skip_serializing_if = "Vec::is_empty")]
1203    pub invalidated: Vec<String>,
1204}
1205
1206/// The derived idempotency key of a forgetting:
1207/// `memory_forget:{scope}:{memory_id}`. Retried erasures converge on one
1208/// key — forgetting an already-forgotten record is the same journaled
1209/// effect, not a second one.
1210pub fn memory_forget_effect_key(scope: &ScopeAddress, memory_id: &str) -> String {
1211    format!("memory_forget:{}:{memory_id}", scope.as_address())
1212}
1213
1214/// What a forgetting will do, computed over the namespace **before**
1215/// anything is deleted: the explicitly forgotten records and the dependent
1216/// summaries the erasure invalidates. Planning is a pure function of store
1217/// state, so the route can journal exactly what it is about to do — the
1218/// tombstone names this plan, and the deletion then executes it.
1219#[derive(Debug, Clone, Default, PartialEq, Eq)]
1220pub struct ForgetPlan {
1221    /// The explicitly forgotten records (the targets that exist in the
1222    /// namespace), sorted for determinism.
1223    pub forgotten: Vec<String>,
1224
1225    /// The dependent summaries invalidated by the erasure, sorted: every
1226    /// `summary` record that names a removed record in
1227    /// [`MemoryEvidence::source_memory_ids`], transitively (a summary of
1228    /// summaries goes too). They are deleted with the targets — a summary
1229    /// built on erased evidence is derived state, and leaving it serving
1230    /// content distilled from the forgotten record is not forgetting
1231    /// (Cao & Yang's point, per the design).
1232    pub invalidated: Vec<String>,
1233}
1234
1235/// Plan a forgetting over `universe` (one tenant's namespace): `targets`
1236/// are the records to erase; the dependent-summary walk runs in reverse
1237/// over source naming until the closure stops growing. Targets absent from
1238/// the universe are skipped (the caller decides whether absence is an
1239/// error — the single-id route 404s before planning).
1240pub fn plan_forget(universe: &[MemoryRecord], targets: &[String]) -> ForgetPlan {
1241    let mut removed: HashSet<&str> = universe
1242        .iter()
1243        .filter(|record| targets.iter().any(|t| t == &record.memory_id))
1244        .map(|record| record.memory_id.as_str())
1245        .collect();
1246    // The fixpoint: a summary joins the removal set whenever it names a
1247    // record already in it. Terminates because the set grows monotonically
1248    // over a finite universe.
1249    loop {
1250        let mut grew = false;
1251        for record in universe {
1252            if record.kind != MemoryKind::Summary || removed.contains(record.memory_id.as_str()) {
1253                continue;
1254            }
1255            if record
1256                .provenance
1257                .evidence
1258                .source_memory_ids
1259                .iter()
1260                .any(|source| removed.contains(source.as_str()))
1261            {
1262                removed.insert(record.memory_id.as_str());
1263                grew = true;
1264            }
1265        }
1266        if !grew {
1267            break;
1268        }
1269    }
1270    let mut forgotten: Vec<String> = removed
1271        .iter()
1272        .filter(|id| targets.iter().any(|t| t.as_str() == **id))
1273        .map(|id| id.to_string())
1274        .collect();
1275    forgotten.sort();
1276    let mut invalidated: Vec<String> = removed
1277        .iter()
1278        .filter(|id| !forgotten.iter().any(|f| f.as_str() == **id))
1279        .map(|id| id.to_string())
1280        .collect();
1281    invalidated.sort();
1282    ForgetPlan {
1283        forgotten,
1284        invalidated,
1285    }
1286}
1287
1288// --------------------------------------------------------------------- //
1289// Conflict detection (R0.8 wave 2): evidence, never resolution
1290// --------------------------------------------------------------------- //
1291
1292/// A flagged conflict: two live records that share a key, overlap in
1293/// validity, and carry contradictory content. A review item — the design's
1294/// rule is that detection is evidence and resolution is governance, so
1295/// nothing here (or anywhere in the runtime) resolves the pair: Zep and
1296/// Mem0 resolve contradictions inside the ingestion pipeline with an LLM,
1297/// which is precisely the silent-mutation pattern the learning rule exists
1298/// to forbid.
1299#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
1300pub struct MemoryConflict {
1301    /// The scope the conflicted records live at.
1302    pub scope: ScopeAddress,
1303
1304    /// The key they share.
1305    pub key: String,
1306
1307    /// The two records' content addresses, sorted.
1308    pub memory_ids: Vec<String>,
1309
1310    /// The interval both records claim to be true over — the overlap of
1311    /// their validity windows.
1312    pub overlap: ValidityWindow,
1313}
1314
1315/// Flag the conflicts in `universe` (one tenant's namespace) at `now`.
1316///
1317/// "Live" excludes exactly what default retrieval excludes: superseded
1318/// records (a superseded record is replaced evidence, not a contender) and
1319/// records expired at `now`. "Contradictory" is structural and honest about
1320/// it: with no semantic judge in the runtime, contradiction means the two
1321/// records assert *different canonical content* for the same key over an
1322/// overlapping interval — semantics belong to the reviewer the flag is for.
1323/// A pair where one record supersedes the other is disciplined replacement,
1324/// not conflict. The output is deterministically ordered (scope, key, ids),
1325/// so equal store state flags byte-equal review items.
1326pub fn detect_conflicts(universe: &[MemoryRecord], now: DateTime<Utc>) -> Vec<MemoryConflict> {
1327    let superseded = superseded_set(universe);
1328    let is_live = |record: &MemoryRecord| {
1329        !superseded.contains(record.memory_id.as_str())
1330            && record.expires_at.is_none_or(|expires| expires > now)
1331    };
1332    let mut groups: BTreeMap<(String, &str), Vec<&MemoryRecord>> = BTreeMap::new();
1333    for record in universe.iter().filter(|r| is_live(r)) {
1334        let Some(key) = record.key.as_deref() else {
1335            continue;
1336        };
1337        groups
1338            .entry((record.scope.as_address(), key))
1339            .or_default()
1340            .push(record);
1341    }
1342    let mut conflicts = Vec::new();
1343    for ((_, key), mut group) in groups {
1344        group.sort_by(|a, b| a.memory_id.cmp(&b.memory_id));
1345        for (i, a) in group.iter().enumerate() {
1346            for b in &group[i + 1..] {
1347                if a.supersedes.as_deref() == Some(b.memory_id.as_str())
1348                    || b.supersedes.as_deref() == Some(a.memory_id.as_str())
1349                {
1350                    continue;
1351                }
1352                let Some(overlap) = validity_overlap(&a.validity, &b.validity) else {
1353                    continue;
1354                };
1355                if same_content(a, b) {
1356                    continue;
1357                }
1358                conflicts.push(MemoryConflict {
1359                    scope: a.scope.clone(),
1360                    key: key.to_string(),
1361                    memory_ids: vec![a.memory_id.clone(), b.memory_id.clone()],
1362                    overlap,
1363                });
1364            }
1365        }
1366    }
1367    conflicts.sort_by(|a, b| {
1368        a.scope
1369            .as_address()
1370            .cmp(&b.scope.as_address())
1371            .then_with(|| a.key.cmp(&b.key))
1372            .then_with(|| a.memory_ids.cmp(&b.memory_ids))
1373    });
1374    conflicts
1375}
1376
1377/// The intersection of two validity windows: `[max(from), min(until))`,
1378/// `None` when the windows are disjoint. Open-endedness composes (open ∩
1379/// closed is the closed end; open ∩ open stays open).
1380fn validity_overlap(a: &ValidityWindow, b: &ValidityWindow) -> Option<ValidityWindow> {
1381    let valid_from = a.valid_from.max(b.valid_from);
1382    let valid_until = match (a.valid_until, b.valid_until) {
1383        (Some(x), Some(y)) => Some(x.min(y)),
1384        (x, y) => x.or(y),
1385    };
1386    if valid_until.is_some_and(|until| valid_from >= until) {
1387        return None;
1388    }
1389    Some(ValidityWindow {
1390        valid_from,
1391        valid_until,
1392    })
1393}
1394
1395/// Content equality by canonical hash, so inline and artifact-referenced
1396/// forms of the same body compare equal (the payload discipline's rule
1397/// applied to conflict detection).
1398fn same_content(a: &MemoryRecord, b: &MemoryRecord) -> bool {
1399    match (a.content.content_hash(), b.content.content_hash()) {
1400        (Ok(x), Ok(y)) => x == y,
1401        _ => a.content == b.content,
1402    }
1403}
1404
1405// --------------------------------------------------------------------- //
1406// Consolidation (R0.8 wave 2): the runtime-owned summary invariants
1407// --------------------------------------------------------------------- //
1408
1409/// Build the one `summary` record that distills `sources`, under the
1410/// invariants the runtime owns: the sources are named in
1411/// [`MemoryEvidence::source_memory_ids`] (which supersedes them — see
1412/// [`apply_query`] — and makes dependent-summary invalidation computable on
1413/// forgetting), the author is `distiller:{name}`, and the confidence is the
1414/// **minimum of the sources'**: a summary is a claim no stronger than its
1415/// weakest source, and the model is honest about that.
1416///
1417/// What the runtime deliberately does not own: `content`. The distillation
1418/// semantics are application code (open question 2's boundary, the same one
1419/// the distiller contract draws); this function is the orchestration around
1420/// them. The record's validity spans the sources (earliest `valid_from`;
1421/// the latest `valid_until` when every source is closed, open-ended
1422/// otherwise — a summary cannot outclaim the evidence it distills).
1423///
1424/// Fails when `sources` is empty: a summary that names no sources is not a
1425/// consolidation, and an unnamed derivation cannot be audited or forgotten
1426/// correctly.
1427pub fn consolidation_summary(
1428    scope: ScopeAddress,
1429    distiller: impl Into<String>,
1430    sources: &[MemoryRecord],
1431    content: Value,
1432    now: DateTime<Utc>,
1433) -> Result<MemoryRecord> {
1434    if sources.is_empty() {
1435        return Err(invalid(
1436            "consolidation needs at least one source record — a summary that names no \
1437             sources is not a consolidation, and an unnamed derivation can neither be \
1438             audited nor forgotten correctly",
1439        ));
1440    }
1441    let mut source_ids: Vec<String> = sources
1442        .iter()
1443        .map(|record| record.memory_id.clone())
1444        .collect();
1445    source_ids.sort();
1446    let confidence = sources
1447        .iter()
1448        .map(|record| record.confidence)
1449        .fold(f64::INFINITY, f64::min);
1450    let valid_from = sources
1451        .iter()
1452        .map(|record| record.validity.valid_from)
1453        .min()
1454        .expect("sources is non-empty");
1455    let valid_until = sources
1456        .iter()
1457        .map(|record| record.validity.valid_until)
1458        .collect::<Option<Vec<DateTime<Utc>>>>()
1459        .and_then(|untils| untils.into_iter().max());
1460    let provenance = MemoryProvenance {
1461        author: ProvenanceAuthor::Distiller {
1462            name: distiller.into(),
1463        },
1464        evidence: MemoryEvidence {
1465            source_memory_ids: source_ids,
1466            ..MemoryEvidence::default()
1467        },
1468        written_at: now,
1469    };
1470    MemoryRecord::new(
1471        MemoryKind::Summary,
1472        scope,
1473        provenance,
1474        confidence,
1475        ValidityWindow {
1476            valid_from,
1477            valid_until,
1478        },
1479        now,
1480        content,
1481    )
1482}
1483
1484// --------------------------------------------------------------------- //
1485// The store contract
1486// --------------------------------------------------------------------- //
1487
1488/// A governed memory store. The two server backends (JSON files, Postgres)
1489/// implement it over their own layouts; [`InMemoryMemoryStore`] is the
1490/// dev/test implementation, the same role `InMemoryCheckpointer` plays for
1491/// checkpoints.
1492///
1493/// The contract deliberately knows nothing about tenants: tenant isolation
1494/// is the server's `{tenant}/` id-namespacing, applied by the server
1495/// backends around this contract — a store that can see another tenant's
1496/// records is a breach wearing a storage costume.
1497#[async_trait]
1498pub trait MemoryStore: Send + Sync + std::fmt::Debug {
1499    /// Store `record` under its content address. Idempotent: storing the
1500    /// same record twice yields `false` (already present) and at most one
1501    /// stored record — the `Effect::Idempotent` write converges by
1502    /// construction, since the address is derived from content plus
1503    /// provenance.
1504    async fn put(&self, record: &MemoryRecord) -> Result<bool>;
1505
1506    /// Fetch one record by content address (`None` when absent).
1507    async fn get(&self, memory_id: &str) -> Result<Option<MemoryRecord>>;
1508
1509    /// The query universe: every record in the namespace. Retrieval
1510    /// semantics live in [`apply_query`]; dev-scale stores scan, and larger
1511    /// backends pre-filter on columns before applying the same function.
1512    async fn all(&self) -> Result<Vec<MemoryRecord>>;
1513
1514    /// Remove `memory_id` from the store (`false` when absent). Forgetting
1515    /// (R0.8 wave 2) is real deletion of derived state — journals are
1516    /// hash-chained evidence and are never touched (open question 4's
1517    /// boundary).
1518    async fn remove(&self, memory_id: &str) -> Result<bool>;
1519
1520    /// The records matching `query`, expiry evaluated at `now`. The default
1521    /// implementation scans via [`MemoryStore::all`] and applies
1522    /// [`apply_query`]; backends with column-mapped storage override the
1523    /// scan, never the semantics.
1524    async fn query(&self, query: &MemoryQuery, now: DateTime<Utc>) -> Result<Vec<MemoryRecord>> {
1525        Ok(apply_query(&self.all().await?, query, now))
1526    }
1527}
1528
1529/// In-memory [`MemoryStore`] (dev and test): one map keyed by content
1530/// address. No persistence, no tenant concept — the honest reference for
1531/// the contract's semantics.
1532#[derive(Debug, Default)]
1533pub struct InMemoryMemoryStore {
1534    records: Mutex<BTreeMap<String, MemoryRecord>>,
1535}
1536
1537impl InMemoryMemoryStore {
1538    /// An empty store.
1539    pub fn new() -> Self {
1540        Self::default()
1541    }
1542
1543    fn lock(&self) -> MutexGuard<'_, BTreeMap<String, MemoryRecord>> {
1544        // Poison means a store path panicked mid-write; the map is plain
1545        // data and stays coherent, so recovering is safe.
1546        self.records.lock().unwrap_or_else(|e| e.into_inner())
1547    }
1548}
1549
1550#[async_trait]
1551impl MemoryStore for InMemoryMemoryStore {
1552    async fn put(&self, record: &MemoryRecord) -> Result<bool> {
1553        let mut map = self.lock();
1554        if map.contains_key(&record.memory_id) {
1555            return Ok(false);
1556        }
1557        map.insert(record.memory_id.clone(), record.clone());
1558        Ok(true)
1559    }
1560
1561    async fn get(&self, memory_id: &str) -> Result<Option<MemoryRecord>> {
1562        Ok(self.lock().get(memory_id).cloned())
1563    }
1564
1565    async fn all(&self) -> Result<Vec<MemoryRecord>> {
1566        Ok(self.lock().values().cloned().collect())
1567    }
1568
1569    async fn remove(&self, memory_id: &str) -> Result<bool> {
1570        Ok(self.lock().remove(memory_id).is_some())
1571    }
1572}
1573
1574// --------------------------------------------------------------------- //
1575// The journaled seam
1576// --------------------------------------------------------------------- //
1577
1578/// The canonical JSON shape a [`JournaledMemory`] read journals (and a
1579/// [`MemoryReplaySource`] matches) as a memory read's **request**: the
1580/// resolved query (with [`MemoryQuery::as_of`] stamped) and the budget,
1581/// exactly as passed to the store. The hash of this value's canonical
1582/// serialization is the request identity exact replay matches on — the same
1583/// rule [`crate::replay::model_call_request`] sets for model calls.
1584pub fn memory_read_request(query: &MemoryQuery, budget: &ContextBudget) -> Value {
1585    json!({ "query": query, "budget": budget })
1586}
1587
1588/// The derived idempotency key of a memory write:
1589/// `memory:{scope}:{memory_id}` (the design's write-path rule). Retried
1590/// submissions of the same write converge on one key, and recovery can
1591/// re-derive it without consulting the store.
1592pub fn memory_effect_key(scope: &ScopeAddress, memory_id: &str) -> String {
1593    format!("memory:{}:{memory_id}", scope.as_address())
1594}
1595
1596/// Canonical-content hash of a request value — the same computation
1597/// `PayloadRef::content_hash` applies and the same one
1598/// [`crate::replay::ReplaySource`] matches model/tool requests with.
1599fn request_hash(request: &Value) -> Result<String> {
1600    Ok(sha256_hex(&serde_json::to_vec(request)?))
1601}
1602
1603#[derive(Debug)]
1604struct MemoryReplayInner {
1605    /// The snapshot's `MemoryRead` events, in `seq` order.
1606    servable: Vec<RunEvent>,
1607    /// The snapshot's artifact map, for payload resolution.
1608    artifacts: BTreeMap<String, Value>,
1609    /// Position of the next unserved event. Strictly ordered, the same rule
1610    /// the Flight Recorder's replay source applies.
1611    cursor: usize,
1612}
1613
1614/// The serving side of exact replay for memory reads: an ordered cursor
1615/// over a journaled run's recorded [`RunEventKind::MemoryRead`] events.
1616///
1617/// This is [`crate::replay::ReplaySource`] specialized to one event kind —
1618/// the same matching rule (sequence + request hash), the same loud failure
1619/// on any mismatch — kept in this module because a memory read is answered
1620/// from the memory seam, not from a model or tool wrapper. Matching is by
1621/// **sequence + request hash**: [`MemoryReplaySource::serve`] takes the next
1622/// unserved `MemoryRead` event in `seq` order and requires the issued
1623/// request's canonical hash to equal the journaled request's. Anything else
1624/// fails with [`RustyError::Replay`] — a replay that has drifted from its
1625/// evidence must stop, not improvise.
1626#[derive(Debug, Clone)]
1627pub struct MemoryReplaySource {
1628    inner: Arc<Mutex<MemoryReplayInner>>,
1629}
1630
1631impl MemoryReplaySource {
1632    /// A source over the `MemoryRead` events of `snapshot`. The snapshot is
1633    /// assumed already integrity-verified (the caller's replay session
1634    /// verifies it); unresolved artifact references simply fail to match.
1635    pub fn new(snapshot: &JournalSnapshot) -> Self {
1636        Self {
1637            inner: Arc::new(Mutex::new(MemoryReplayInner {
1638                servable: snapshot
1639                    .events
1640                    .iter()
1641                    .filter(|event| event.kind == RunEventKind::MemoryRead)
1642                    .cloned()
1643                    .collect(),
1644                artifacts: snapshot.artifacts.clone(),
1645                cursor: 0,
1646            })),
1647        }
1648    }
1649
1650    fn lock(&self) -> MutexGuard<'_, MemoryReplayInner> {
1651        self.inner.lock().unwrap_or_else(|e| e.into_inner())
1652    }
1653
1654    /// Serve one memory read from the journal: the next unserved
1655    /// `MemoryRead` event whose journaled request hash-equals `request`.
1656    /// The returned [`ServedEffect`] carries the recorded assembly; re-journal
1657    /// it into the replay run's journal so the replayed evidence reproduces
1658    /// the recorded evidence byte-for-byte.
1659    pub fn serve(&self, request: &Value) -> Result<ServedEffect> {
1660        let issued_hash = request_hash(request)?;
1661        let mut inner = self.lock();
1662        let Some(event) = inner.servable.get(inner.cursor) else {
1663            return Err(replay_error(format!(
1664                "replay exhaustion: the run issued a memory read (request hash {issued_hash}), \
1665                 but the journal holds no further recorded memory reads to serve — the \
1666                 replayed run is doing more work than the recorded one"
1667            )));
1668        };
1669        let recorded_input = event
1670            .input
1671            .as_ref()
1672            .and_then(|payload| match payload {
1673                PayloadRef::Inline(value) => Some(value.clone()),
1674                PayloadRef::Artifact(reference) => inner.artifacts.get(&reference.sha256).cloned(),
1675            })
1676            .ok_or_else(|| {
1677                replay_error(format!(
1678                    "recorded memory read at seq {} has no input payload to match against",
1679                    event.seq
1680                ))
1681            })?;
1682        let recorded_hash = request_hash(&recorded_input)?;
1683        if recorded_hash != issued_hash {
1684            return Err(replay_error(format!(
1685                "replay divergence at recorded seq {} (memory read): the run issued a request \
1686                 whose canonical hash is {issued_hash}, but the journaled request hashes to \
1687                 {recorded_hash} — the replayed run has diverged from its evidence",
1688                event.seq
1689            )));
1690        }
1691        let served = ServedEffect {
1692            input: Some(recorded_input),
1693            output: event.output.as_ref().and_then(|payload| match payload {
1694                PayloadRef::Inline(value) => Some(value.clone()),
1695                PayloadRef::Artifact(reference) => inner.artifacts.get(&reference.sha256).cloned(),
1696            }),
1697            event: event.clone(),
1698        };
1699        inner.cursor += 1;
1700        Ok(served)
1701    }
1702
1703    /// `true` when every recorded memory read has been served. Verification
1704    /// treats a non-exhausted source as a replay that stopped short of its
1705    /// evidence.
1706    pub fn is_exhausted(&self) -> bool {
1707        let inner = self.lock();
1708        inner.cursor == inner.servable.len()
1709    }
1710}
1711
1712/// Where a journaled memory handle answers reads from: the live store
1713/// (record mode) or a recorded journal (replay mode).
1714///
1715/// Node code holds one [`JournaledMemory`] either way and cannot tell the
1716/// difference — the same discipline the Flight Recorder's recording and
1717/// replaying model wrappers apply: behavior that matters is journaled, and
1718/// replay serves the journal instead of the world.
1719#[derive(Debug, Clone)]
1720pub enum MemorySource {
1721    /// Read from and write to a live [`MemoryStore`], journaling every
1722    /// operation.
1723    Store(Arc<dyn MemoryStore>),
1724
1725    /// Read from a recorded journal ([`MemoryReplaySource`]), re-journaling
1726    /// the served events byte-identically. Writes fail: a replayed run
1727    /// issuing a write has diverged from its evidence.
1728    Replay(MemoryReplaySource),
1729}
1730
1731/// A memory handle bound to a run's journal: the journaled read/write seam
1732/// node code (and later waves' run integration) uses.
1733///
1734/// Cheap to clone (the journal and source are shared handles), so node
1735/// closures capture it exactly the way they capture a [`Journal`] for
1736/// recording model and tool calls. Causal parentage comes from the
1737/// invocation's node-input event id, passed as `parent` — node code reads
1738/// it from [`crate::journal::PARENT_EVENT_KEY`] in `NodeConfig::extra`.
1739///
1740/// Clock-read parity: the live and replay paths read the run's clock the
1741/// same number of times in the same order (one `as_of` stamp, two latency
1742/// reads, one read inside [`Journal::record`]), so a logical clock's tick
1743/// sequence — and therefore every journaled timestamp — reproduces exactly
1744/// on replay. This is the same parity rule the replaying model wrapper
1745/// documents.
1746#[derive(Debug, Clone)]
1747pub struct JournaledMemory {
1748    journal: Journal,
1749    source: MemorySource,
1750}
1751
1752impl JournaledMemory {
1753    /// Bind `source` to `journal` (the run's journal — build via
1754    /// [`Journal::memory`]).
1755    pub fn new(journal: &Journal, source: MemorySource) -> Self {
1756        Self {
1757            journal: journal.clone(),
1758            source,
1759        }
1760    }
1761
1762    /// The source this handle answers from.
1763    pub fn source(&self) -> &MemorySource {
1764        &self.source
1765    }
1766
1767    /// Perform a governed read: resolve [`MemoryQuery::as_of`] through the
1768    /// run's clock when unset, then — live — query the store, assemble
1769    /// deterministically, and journal the assembly as a
1770    /// [`RunEventKind::MemoryRead`] event ([`Effect::ReadOnly`]); or —
1771    /// replay — serve the journaled assembly byte-identically and re-journal
1772    /// it. Returns the assembly either way.
1773    pub async fn read(
1774        &self,
1775        query: &MemoryQuery,
1776        budget: &ContextBudget,
1777        parent: Option<String>,
1778    ) -> Result<MemoryAssembly> {
1779        // The resolved query fully determines the result: as_of stamped
1780        // through the run's clock (deterministic under a logical clock),
1781        // everything else declared. One clock read here in both paths —
1782        // the parity the byte-identical replay property rests on.
1783        let mut resolved = query.clone();
1784        let as_of = resolved.as_of.unwrap_or_else(|| self.journal.clock().now());
1785        resolved.as_of = Some(as_of);
1786        let request = memory_read_request(&resolved, budget);
1787        match &self.source {
1788            MemorySource::Store(store) => {
1789                let started = self.journal.clock().now();
1790                let records = store.query(&resolved, as_of).await?;
1791                let latency_ms = (self.journal.clock().now() - started)
1792                    .num_milliseconds()
1793                    .max(0) as u64;
1794                let assembly = assemble(records, budget)?;
1795                let mut draft = EventDraft::new(RunEventKind::MemoryRead, Effect::ReadOnly)
1796                    .input(request)
1797                    .output(serde_json::to_value(&assembly)?)
1798                    .latency_ms(latency_ms);
1799                if let Some(parent) = parent {
1800                    draft = draft.parent(parent);
1801                }
1802                self.journal.record(draft);
1803                Ok(assembly)
1804            }
1805            MemorySource::Replay(source) => {
1806                let served = source.serve(&request)?;
1807                // Clock-read parity with the live path (two latency reads
1808                // per read) keeps the logical clock's tick sequence aligned
1809                // with the recorded run, so the replayed journal's
1810                // timestamps reproduce it exactly.
1811                let _started = self.journal.clock().now();
1812                let _ended = self.journal.clock().now();
1813                let parent = parent
1814                    .or_else(|| served.event.parent.clone())
1815                    .unwrap_or_default();
1816                served.rejournal(&self.journal, parent);
1817                let output = served.output.clone().ok_or_else(|| {
1818                    replay_error(format!(
1819                        "recorded memory read at seq {} carries no assembly payload — the \
1820                         journal is inconsistent",
1821                        served.event.seq
1822                    ))
1823                })?;
1824                let assembly: MemoryAssembly = serde_json::from_value(output)?;
1825                Ok(assembly)
1826            }
1827        }
1828    }
1829
1830    /// Perform a governed write: store the record (content-addressed, so
1831    /// retries converge) and journal it as a [`RunEventKind::MemoryWrite`]
1832    /// event — [`Effect::Idempotent`] under the derived key
1833    /// `memory:{scope}:{memory_id}`, with the record and its provenance as
1834    /// the output payload. Returns the content address.
1835    ///
1836    /// A memory write changes what future retrievals return — nothing else
1837    /// (the write path's third gate). If the write is meant to change a
1838    /// prompt, a policy, or a permission, it belongs in the candidate
1839    /// pipeline, not here.
1840    ///
1841    /// Fails during replay: writes are never served, so a replayed run
1842    /// issuing one has diverged from its evidence.
1843    pub async fn write(&self, record: &MemoryRecord, parent: Option<String>) -> Result<String> {
1844        match &self.source {
1845            MemorySource::Store(store) => {
1846                store.put(record).await?;
1847                let effect_key = memory_effect_key(&record.scope, &record.memory_id);
1848                let mut draft = EventDraft::new(RunEventKind::MemoryWrite, Effect::Idempotent)
1849                    .input(json!({
1850                        "effect_key": effect_key,
1851                        "memory_id": record.memory_id,
1852                    }))
1853                    .output(serde_json::to_value(record)?);
1854                if let Some(parent) = parent {
1855                    draft = draft.parent(parent);
1856                }
1857                self.journal.record(draft);
1858                Ok(record.memory_id.clone())
1859            }
1860            MemorySource::Replay(_) => Err(replay_error(
1861                "memory writes are not served during replay: the run issued a write the \
1862                 recorded evidence does not contain — the replayed run has diverged from its \
1863                 evidence",
1864            )),
1865        }
1866    }
1867}
1868
1869#[cfg(test)]
1870mod tests {
1871    use super::*;
1872    use crate::journal::Clock;
1873
1874    fn ts(millis: i64) -> DateTime<Utc> {
1875        DateTime::<Utc>::from_timestamp_millis(millis).unwrap()
1876    }
1877
1878    fn provenance() -> MemoryProvenance {
1879        MemoryProvenance {
1880            author: ProvenanceAuthor::Human {
1881                human_id: "amjad".into(),
1882            },
1883            evidence: MemoryEvidence::default(),
1884            written_at: ts(1_000),
1885        }
1886    }
1887
1888    fn record(content: Value) -> MemoryRecord {
1889        MemoryRecord::new(
1890            MemoryKind::Fact,
1891            ScopeAddress::new(MemoryScope::User, "user-7"),
1892            provenance(),
1893            1.0,
1894            ValidityWindow::starting(ts(500)),
1895            ts(1_000),
1896            content,
1897        )
1898        .unwrap()
1899    }
1900
1901    #[test]
1902    fn scope_mapping_is_the_documented_superset() {
1903        assert_eq!(
1904            MemoryScope::from_state_scope(StateScope::Private),
1905            Some(MemoryScope::Agent)
1906        );
1907        assert_eq!(
1908            MemoryScope::from_state_scope(StateScope::Team),
1909            Some(MemoryScope::Team)
1910        );
1911        assert_eq!(
1912            MemoryScope::from_state_scope(StateScope::User),
1913            Some(MemoryScope::User)
1914        );
1915        assert_eq!(
1916            MemoryScope::from_state_scope(StateScope::Tenant),
1917            Some(MemoryScope::Tenant)
1918        );
1919        assert_eq!(
1920            MemoryScope::Agent.to_state_scope(),
1921            Some(StateScope::Private)
1922        );
1923        assert_eq!(MemoryScope::Run.to_state_scope(), None);
1924        for scope in [
1925            MemoryScope::Run,
1926            MemoryScope::Agent,
1927            MemoryScope::Team,
1928            MemoryScope::User,
1929            MemoryScope::Tenant,
1930        ] {
1931            if let Some(state_scope) = scope.to_state_scope() {
1932                assert_eq!(MemoryScope::from_state_scope(state_scope), Some(scope));
1933            }
1934        }
1935    }
1936
1937    #[test]
1938    fn memory_id_is_a_content_address() {
1939        let a = record(json!({"timezone": "UTC+4"}));
1940        let b = record(json!({"timezone": "UTC+4"}));
1941        let c = record(json!({"timezone": "UTC+1"}));
1942        // Identical content and provenance converge on one id…
1943        assert_eq!(a.memory_id, b.memory_id);
1944        assert_eq!(a.memory_id.len(), 64);
1945        // …changed content is a new id…
1946        assert_ne!(a.memory_id, c.memory_id);
1947        // …and so is changed provenance (origin is identity).
1948        let mut other_provenance = provenance();
1949        other_provenance.written_at = ts(2_000);
1950        let d = MemoryRecord::new(
1951            MemoryKind::Fact,
1952            ScopeAddress::new(MemoryScope::User, "user-7"),
1953            other_provenance,
1954            1.0,
1955            ValidityWindow::starting(ts(500)),
1956            ts(1_000),
1957            json!({"timezone": "UTC+4"}),
1958        )
1959        .unwrap();
1960        assert_ne!(a.memory_id, d.memory_id);
1961    }
1962
1963    #[test]
1964    fn confidence_outside_the_interval_is_rejected() {
1965        for confidence in [0.0, -0.1, 1.01, f64::NAN] {
1966            assert!(
1967                MemoryRecord::new(
1968                    MemoryKind::Fact,
1969                    ScopeAddress::new(MemoryScope::User, "u"),
1970                    provenance(),
1971                    confidence,
1972                    ValidityWindow::starting(ts(0)),
1973                    ts(0),
1974                    json!(null),
1975                )
1976                .is_err(),
1977                "confidence {confidence} must be rejected"
1978            );
1979        }
1980    }
1981
1982    #[test]
1983    fn content_split_follows_the_payload_discipline() {
1984        let small = record(json!({"k": "v"}));
1985        assert!(matches!(small.content, PayloadRef::Inline(_)));
1986        let big = record(json!({"blob": "x".repeat(INLINE_PAYLOAD_MAX_BYTES)}));
1987        let reference = match &big.content {
1988            PayloadRef::Artifact(reference) => reference.clone(),
1989            other => panic!("expected artifact reference, got {other:?}"),
1990        };
1991        assert_eq!(reference.bytes, big.content_bytes());
1992        // The address is derivable from the record itself: identity is
1993        // integrity.
1994        assert_eq!(
1995            derive_memory_id(&big.content, &big.provenance).unwrap(),
1996            big.memory_id
1997        );
1998    }
1999
2000    #[test]
2001    fn estimated_tokens_applies_bytes_divisor_and_margin() {
2002        assert_eq!(estimated_tokens(0, 20), 0);
2003        assert_eq!(estimated_tokens(4, 0), 1);
2004        assert_eq!(estimated_tokens(5, 0), 2); // ceil
2005        assert_eq!(estimated_tokens(400, 20), 120); // 100 * 1.2
2006        assert_eq!(estimated_tokens(u64::MAX, 100), u32::MAX); // saturated
2007    }
2008
2009    #[tokio::test]
2010    async fn journaled_write_is_idempotent_under_the_derived_key() {
2011        let journal = Journal::new("run-1", "thread-1", Clock::System);
2012        let store = Arc::new(InMemoryMemoryStore::new());
2013        let memory = journal.memory(MemorySource::Store(store.clone()));
2014        let record = record(json!({"a": 1}));
2015        let id = memory.write(&record, None).await.unwrap();
2016        assert_eq!(id, record.memory_id);
2017        // A retried submission of the same record converges.
2018        assert!(!store.put(&record).await.unwrap());
2019
2020        let events = journal.events();
2021        assert_eq!(events.len(), 1);
2022        let event = &events[0];
2023        assert_eq!(event.kind, RunEventKind::MemoryWrite);
2024        assert_eq!(event.effect, Effect::Idempotent);
2025        let input = event.input.as_ref().unwrap();
2026        let PayloadRef::Inline(input) = input else {
2027            panic!("write request travels inline")
2028        };
2029        assert_eq!(
2030            input["effect_key"].as_str().unwrap(),
2031            memory_effect_key(&record.scope, &record.memory_id)
2032        );
2033    }
2034
2035    #[tokio::test]
2036    async fn write_fails_during_replay() {
2037        let journal = Journal::new("run-1", "thread-1", Clock::System);
2038        let snapshot = journal.snapshot();
2039        let memory = journal.memory(MemorySource::Replay(MemoryReplaySource::new(&snapshot)));
2040        let err = memory
2041            .write(&record(json!({"a": 1})), None)
2042            .await
2043            .unwrap_err();
2044        assert!(matches!(err, RustyError::Replay(_)));
2045    }
2046}