Module compact 

Compaction + oplog GC (slice B4 of docs/proposals/multi-device-sync.md, §“Deep dive: the checkpoint / compaction / GC protocol”).

Compaction = checkpoint at the stable frontier, then drop the ops below it. Three guards make that safe, each straight from the proposal:

1. The frontier is the min acked HLC over every known device (AckTable::stable_frontier) — “GC of a CRDT log is only safe once every replica has folded past the truncation point”. Compaction refuses to run at all when a device present in the log has no ack entry (CompactError::UnackedDevice) or when nothing is acked (CompactError::NothingAcked): ops above ANY device’s acked frontier are data another device hasn’t seen, and are never dropped. (Device eviction — the horizon H that unpins a dead laptop — is relay policy, B3.)
2. Checkpoint durable FIRST, then truncate (compact_and_truncate). The crash-ordering invariant: a crash after the checkpoint fsync but before the journal rename leaves the full journal plus a redundant checkpoint (harmless — rerunning compaction recomputes the identical content-addressed file); the rename itself is atomic, so mid-truncation crashes leave either the old complete journal or the new complete tail. At no point does acknowledged data exist only in a file that isn’t durably written.
3. Retention is applied to the checkpoint state, never to the tail. The proposal’s per-surface retention table (RetentionPolicy::proposal_default) trims what the snapshot keeps; ops above the frontier are untouched by policy. Replay surfaces (routing observations — path-dependent, replayed from genesis; crate::oplog::Surface::is_replay_stream) additionally reject any rule but keep-all (CompactError::EventStreamRetention), so an unacked (or acked!) observation tail can never be compacted away. (Conversation is an event-stream multiset too — B2 — but its turns are independent, so it tolerates LastN; only replay streams are forbidden.) Every retention-dropped id-bearing entry leaves a minimal tombstone stub ({"id": …, "tombstone": true}, original op_id/hlc kept), so a "supersedes" reference always resolves against a tombstone, not a hole — including a reference that arrives AFTER compaction. (Preserving only the references visible at compaction time was a reproduced divergence: a later tail op superseding an already-dropped entry made the global fold retain what the compacted device could not. Universal stubs are time-hole-free: both sides reduce the same record to the same stub, deterministically. Entries with no entity id — e.g. content-hash-keyed conversation turns — cannot be referenced by id and drop entirely.) Stubs are carried unchanged by later retention passes and never count against a surface’s retention quota.

Determinism note (the “same frontier ⇒ same snapshot hash” invariant): retention is deterministic over a fixed state, and the age-based rules’ reference instant defaults to as_of_from_ops — the max payload "timestamp" among the ops at/below the frontier, i.e. pure over the same inputs the fold already consumes, so every device compacting the same frontier derives the same instant with no out-of-band agreement (this crate never reads a clock; a caller may still pass an explicit Some(as_of_ms)). The derived value is conservative: a stale max under-drops, and undated entries never age-drop anyway. Ordering/recency come from the payload’s numeric "timestamp" field (ms) and grouping from "agent_id"; an entry with no timestamp is treated as newest / never age-dropped — undated data is never silently discarded.

Structs

AckTable

The fold-frontier / ack bookkeeping B3’s relay ack(frontier) reports against: per-device max folded HLC, monotone-only advance, persisted alongside the checkpoint (temp + atomic rename, like everything durable here). This is the proposal’s acked[device] table, device-local.

CompactionOutcome

The outcome of an executed compaction.

CompactionPlan

A computed (not yet executed) compaction: the retained checkpoint, the ops that stay in the journal, and what happened.

RetentionPolicy

Per-surface retention policy: surface tag → rule, with a conservative keep-all default for unknown surfaces. Applies to the grow-only log tier only — the LWW registry tier is already one compact record per id, and the proposal’s table assigns it no retention.

RetentionReport

What retention did: per-surface counts of entries dropped entirely (id-less — nothing can reference them) and the entries reduced to tombstone stubs, as (surface_tag, key).

Enums

CompactError

A compaction failure. No PartialEq (carries io::Error); match on variants.

RetentionRule

How one surface’s checkpoint retention trims — the proposal’s table rows.

Constants

RUNS_MAX_AGE_MS

…and drop runs older than 30 days (in ms), whichever is more restrictive — exactly RunStore::gc’s rule, made globally coherent.

RUNS_MAX_PER_AGENT

Parity constants with today’s run GC (car-server-core::run_store): keep the 50 most recent runs per agent…

Functions

apply_retention

Apply per-surface retention to a folded (checkpoint) state — the proposal’s “the snapshot applies each surface’s retention”, with the §“Two free properties fall out” referential-integrity rule realized as universal tombstone stubs: every dropped id-bearing entry is reduced to {"id", "tombstone": true} (original op_id/hlc kept) rather than erased, so a "supersedes" reference — even one that arrives after compaction — always resolves. Stubs never count against a rule’s quota and are carried unchanged by later passes. Pure over its inputs; as_of_ms is the reference instant for the age rules (no clock reads here — plan_compaction defaults it via as_of_from_ops).

as_of_from_ops

The deterministic default reference instant for age-based retention: the max payload "timestamp" across ops (0 when none carry one). Pure over the same inputs the fold consumes, so every device compacting the same frontier derives the identical value — no out-of-band agreement, no clock read. Conservative by construction: a stale max under-drops (age rules see everything as newer), and undated entries never age-drop regardless.

compact_and_truncate

Execute a compaction end-to-end on a live journal, enforcing the crash-ordering invariant by construction:

is_tombstone

Is this record a retention tombstone stub? ("tombstone": true is the reserved marker apply_retention stamps.)

plan_compaction

Plan a compaction of ops at the AckTable’s stable frontier. Pure — no file IO; compact_and_truncate executes a plan durably. See the module docs for the three safety guards; per-device HLC monotonicity (enforced by verify_log) guarantees “hlc ≤ frontier” is a chain prefix, so the cut is always anchorable.