rostra-client-db 0.1.4

Embedded database and projections for a Rostra client
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
# Event Content Lifecycle

> **See also**: `src/tables.rs` for table definitions and inline documentation.
> When updating this document, ensure `tables.rs` stays in sync.

This document describes how events and their content are tracked, processed, and
reference-counted in the Rostra client database.

## Overview

Events in Rostra form a DAG (Directed Acyclic Graph). Each event has an
"envelope" (metadata + signature) and "content" (payload). These are stored
separately to enable:

1. **Content deduplication**: Same content shared by multiple events is stored once
2. **Content pruning**: Large content can be discarded while keeping DAG structure
3. **Out-of-order delivery**: Events can arrive before their content

The public `Database::process_event_content` method also accepts the reverse
arrival order. `VerifiedEventContent` includes the verified envelope, so the
method inserts a missing envelope and processes the content atomically. Calling
`process_event` before or after it, or delivering either item repeatedly, does
not repeat reference counts, usage accounting, lifecycle transitions, or
content-derived projections. The lower-level `process_event_content_tx` helper
assumes an existing envelope and remains internal for transaction composition
and migration replay.

Content may be empty (`content_len == 0`). Empty content is handled as normal
content — it gets an RC entry and is stored in `content_store` immediately at
event insertion time unless the event starts in Deleted.

New ingestion and total replay enforce the exclusive maximum described below.

## Key Tables

| Table | Key | Purpose |
|-------|-----|---------|
| `events` | `ShortEventId` | Main event storage (envelope only) |
| `content_store` | `ContentHash` | Content storage (deduplicated by hash) |
| `content_rc` | `ContentHash` | Reference count per content hash |
| `events_content_state` | `ShortEventId` | Per-event processing state |
| `events_content_missing` | `(Timestamp, ShortEventId)` | Events waiting for content, sorted by next fetch time |
| `social_posts_by_received_at` | `(Timestamp, u64)` | Social posts ordered by effective local receipt time |
| `social_posts_received_at_keys` | `ShortEventId` | Exact reverse key for removing a social receipt without scanning |
| `social_post_materializations` | `u64` | Append-only ordinary SocialPost materializations in local commit order |

## Total Replay

Opening a database older than schema version 25 performs one total rebuild
before reading derived rows. The preparation transaction stashes signed
envelopes, available immutable content, identity and initialization metadata,
event acquisition sources, and canonical forward replacement lineage. It
preserves caller-owned extension tables and discards every other built-in table.

Replay streams the stash twice in stable event-ID order. The envelope pass first
establishes the complete graph and final deletion, pruning, and missing state.
The content pass then processes available eligible payloads under current
reducers. This phase boundary prevents a deleted target from gaining a
projection merely because its payload replayed first; no parent-before-child
topology is required. Rebuilt receipts preserve source and membership, use
authored timestamps under a uniform policy, and receive new database-local
sequence values. Preparation intentionally discards available envelope receipt
timestamps; content-specific effective receipt times cannot generally be
reconstructed.

Retained records are trusted as previously authenticated ingestion output.
Replay retains typed decoding and payload commitment checks but does not repeat
Ed25519 authentication or claim to audit the database. Failure rolls back the
complete replay transaction and leaves the separately committed stash for the
next open.

Replay does not retain the event graph or per-event commit hooks. Application
and codec code transiently hold the current record, a decoded below-limit
payload, payload clones, and encoding scratch space; follow-history deletion uses
fixed 256-key batches. Work scales with event/reference B-tree operations and
total referenced payload bytes, not only unique content bytes. The final watch
refresh allocates proportional to self followees, followers, and two-hop WoT.

redb also retains dirty-page bookkeeping in process memory and copy-on-write
rollback pages for the whole atomic transaction. Total RAM and peak disk can
therefore scale with database size. There is no resource preflight or safe fixed
free-space multiplier. Measure a production-shaped copy, provision monitored RAM
and disk headroom, retain a restorable backup, and plan a long first open. Replay
does not compact the file.

Schema 26 adds an empty SocialPost materialization feed without backfill. Normal
ingestion appends after a successful ordinary projection in the same
transaction. Total replay suppresses new feed rows and preserves an existing
schema-26 feed byte-for-byte. A scan resolves each retained event identity
against current state, so later deletion, pruning, or replacement returns a
removed marker while leaving the occurrence and sequence intact.

The stacked version-24 schema changes and the version-25 rebuild must ship as one
deployable release. Never deploy an intermediate version-24 ancestor against
production storage. Once preparation commits version 25, rollback requires
restoring the pre-upgrade backup rather than running an older binary.

## State Machine

Each event's content processing goes through these states:

```
                                ┌─────────────────────────────────┐
                                │                                 │
                                v                                 │
Event Inserted ──► Missing ──► (no entry) ──► Deleted/Pruned     │
                      │               │              │            │
                      │               │              └────────────┘
                      │               │              (content delete after
                      │               │               prune/invalid changes
                      │               │               state, no RC change)
                      ├───────────────┴─► Deleted/Pruned
                      │  (content delete/     (if content deleted
                      │   prune before         before event arrives
                      │   content processing)  via events_missing)
                      │
                      └─► Invalid ──► Deleted
                        (content failed     (author content
                         validation)         delete records intent)
```

Note: Events with `content_len == 0` skip the `Missing` state entirely. They go
straight to "no entry" (processed) during event insertion unless they start in
Deleted.

### State Meanings

| State | `events_content_state` | Meaning |
|-------|------------------------|---------|
| Missing | `Missing { last_fetch_attempt, fetch_attempt_count, next_fetch_attempt }` | Event inserted, content not yet processed |
| Processed | *no entry* | Content processed, side effects applied |
| Invalid | `Invalid` | Content failed validation (e.g. CBOR deserialization) |
| Deleted | `Deleted { deleted_by }` | Author deleted this content |
| Pruned | `Pruned` | Locally pruned (too large, etc.) |

**Key insight**: "No entry" in `events_content_state` means content was
successfully processed. This is the normal state for most events.

## Reference Counting

RC tracks how many events want a particular content hash. This enables garbage
collection when no events need the content.

### RC Rules

1. **Increment**: When event is inserted (all events whose content is not
   already marked for deletion, including `content_len == 0`)
2. **Decrement**: When event content is deleted, pruned, or marked invalid
3. **Never double-decrement**: Guards check if content already Deleted/Pruned/Invalid

### RC and Content Store

- RC is managed at **event insertion time**, not when content arrives
- Content is stored in `content_store` when first processed (or immediately
  for `content_len == 0` events that do not start in Deleted)
- When RC reaches 0, content *can* be garbage collected (not automatic)

## Detailed Flows

### Flow 1: Normal Event Arrival (content_len > 0)

The configured maximum content length is an exclusive upper bound. Content
whose length is below the maximum follows this flow; content at or above it is
pruned during envelope processing and never enters Missing.

```
1. insert_event_tx:
   - Add event to `events`
   - Increment RC for content_hash
   - Mark as Missing { count: 0, next: ZERO } in `events_content_state`
   - Content already in store? Skip adding to `events_content_missing`
   - Otherwise add (Timestamp::ZERO, event_id) to `events_content_missing`

2. process_event_content_tx:
   - Check length eligibility and can_insert_event_content_tx: Missing? → proceed
   - Leave fetch scheduling unchanged if the payload is ineligible
   - Apply side effects (reply counts, follow updates, etc.)
   - Store content in `content_store` (if not already there)
   - Remove from `events_content_missing` (using next_fetch_attempt from state)
   - Remove Missing marker from `events_content_state`
```

### Flow 1b: Non-deleted Empty Content Event (content_len == 0)

```
1. insert_event_tx:
   - Add event to `events`
   - Increment RC for content_hash (blake3 hash of empty bytes)
   - Store empty content in `content_store` (if not already there)
   - Track payload as processed immediately (no Missing state)
   - No entry in `events_content_state` (already "processed")
```

### Flow 2: Event Arrives Before Content

```
1. insert_event_tx (event only):
   - Add event to `events`
   - Increment RC
   - Mark as Missing
   - Add to `events_content_missing` (content not in store)

2. Later, content arrives via another event:
   - Content stored in `content_store`

3. process_event_content_tx:
   - Check Missing → proceed
   - Apply side effects
   - Remove from `events_content_missing`
   - Remove Missing marker
```

### Flow 2b: Content Arrives at the Public Boundary Before the Envelope

```
1. process_event_content receives VerifiedEventContent:
   - Insert its carried verified envelope
   - Establish Missing, Deleted, or Pruned state using normal envelope rules
   - Process eligible content in the same transaction

2. The envelope later arrives separately:
   - Event insertion reports AlreadyPresent
   - RC, usage accounting, scheduling, and projections remain unchanged
```

This path is semantically the same as `process_event_with_content`. If the
carried envelope was already inserted, only eligible Missing content is
processed. Processed, Deleted, Pruned, and Invalid states retain their usual
idempotent behavior.

Latest-event projections apply one order while processing side effects:
follow/unfollow state, profiles, generic singletons, and individual votes keep
the maximum `(event.timestamp, ShortEventId)`. The vote aggregate changes only
when the same candidate wins the individual-vote comparison. Equal-second
events therefore converge independently of payload delivery order.

An individual-vote singleton retains its source event ID, full target, and
authoritative current-projection `Down`, `Neutral`, or `Up` value. Replacement
computes the aggregate delta from that inline old projection and updates every
affected aggregate and the winner in the same transaction. When two full
targets share the shortened event ID used as the singleton auxiliary key, a
winning replacement transfers the contribution between their aggregates and
vote reads return a value only for the retained full target. Reads and
replacement do not resolve the old source payload, so a source legitimately
removed after delete, prune, or garbage collection does not make the retained
projection unavailable.
Only singleton-shaped `SOCIAL_VOTE` events whose auxiliary key matches the
payload target enter this coupled winner/aggregate projection. A missing inline
projection or cached target outside its shortened row key is corruption and
fails closed.
Retained signed source remains authoritative for total replay and explicit
audits. A detected source/cache mismatch requires quarantine or recomputation
of the affected projection rather than changing only the cached winner value.

Active follows also retain the latest unfollow as an exclusive epoch boundary
and retain processed follow-event orders. `first_ts` is the timestamp of the
earliest follow strictly after that boundary, so late follow delivery can move
the current epoch's start earlier without allowing a pre-unfollow follow into a
later epoch. Follow/unfollow events in the same second use `ShortEventId` to
decide epoch membership. Notification cutoff remains timestamp-only: a social
post or shoutbox timestamp below both database initialization and `first_ts`
uses its authored time in the receipt index; one equal to `first_ts` uses local
receipt time.

### Flow 3: Content Deletion Before Target Event Arrives

Parent IDs resolve only within the deleting event's author graph, as specified
by [SPEC-event-graph](https://github.com/dpc/rostra/blob/master/crates/rostra-core/specs/SPEC-event-graph.md).
A present event by another author with the same short ID follows this
missing-target flow and cannot be mutated.

```
1. Delete event D arrives, same-author target T not in `events`:
   - T added to `events_missing` with deleted_by = D
   - An ordinary child referencing T preserves deleted_by = D
   - Another direct deleter is merged canonically; the maximum
     `(event.timestamp, ShortEventId)` becomes deleted_by

2. Same-author target event T finally arrives:
   - Check `events_missing`: found with deleted_by
   - Mark T's content as Deleted in `events_content_state`
   - Do NOT increment RC (content already marked for deletion)
   - Account T's payload directly in total and deleted usage without first
     entering Missing
   - Do NOT mark content as Missing
```

Deletion intent is monotone in both `events_missing` and
`events_content_state`. Multiple direct deleting children select the same
canonical `deleted_by` regardless of delivery order. Changing only the winner
does not repeat RC, queue, usage, or projection-reversion work.

Deletion affects content, not the signed header. In the direct chain
`T <-delete- D1 <-delete- D2`, D1's header keeps deleting T even after D1's own
content is deleted. The direct attributions remain `T.deleted_by = D1` and
`D1.deleted_by = D2`; D2 is not transitively assigned to T.

### Flow 4: Content Deletion After Target (While Content Missing)

```
1. Event T arrives in its author's graph:
   - RC = 1
   - T's content = Missing

2. Same-author delete event D arrives targeting T's content:
   - old_state = Missing
   - Set T's content = Deleted
   - Decrement RC (now 0)

3. Content for T arrives:
   - can_insert_event_content_tx: T's content = Deleted → return false
   - Ordinary content processing is skipped
   - A below-limit SOCIAL_POST with the delete-aux flag, an auxiliary
     parent, valid CBOR, and nonempty `djot_content.trim()` may record only its
     immutable forward and reverse replacement rows
```

The same limited replacement extraction applies when the deletion arrived
before the target envelope. In an edit chain `E <- A <- B`, delivery of B can
stage A as Deleted; later verified A content still records `E -> A`, so lookup
of E follows `E -> A -> B`. Blank deletion posts do not record an edit edge.
Missing, empty, or whitespace-only `djot_content` is blank regardless of other
social-post fields. The database also checks already-retained hash-keyed bytes
when a predeleted envelope arrives, but it never schedules or requests Deleted
content. Total replay preserves the immutable forward row and reconstructs the
reverse index without depending on zero-RC payload bytes; already-retained
bytes remain independently GC-eligible. Deleted content never becomes visible
or retrievable and does not update RC, queues, usage, time/reception indexes,
reply/reaction counts, news, mentions, votes, singletons, or notifications.

For an ordinarily processed SOCIAL_POST, insertion and later deletion
reversion use the same projection-applicability classification:

- a post without the delete-aux flag applies ordinary projections;
- a post with the delete-aux flag but no auxiliary parent applies no ordinary
  projections, even when its body is nonblank;
- a deleting post with an auxiliary parent and nonempty trimmed
  `djot_content` is an edit and applies ordinary projections;
- a deleting post with absent, empty, or whitespace-only `djot_content`
  applies no ordinary projections, even when reply, reaction, or news fields
  are populated.

The symmetric rule covers the authored-time, reply, reaction, news, and
self-mention projections, plus the effective-reception forward row and its
event-to-key reverse mapping. The receipt directions are inserted and removed
atomically. Reversion leaves the shared durable reception allocator advanced;
deleted sequence values are never reused. An absent reverse key is a no-op when
deletion ordering prevented ordinary projection insertion. A present reverse
key whose forward row is absent or names another event is an invariant failure
and rolls back deletion.
Reversion likewise does not compensate with saturating arithmetic: an eligible
projection with inconsistent counters remains an invariant failure. Immutable
replacement lineage follows its separate, Deleted-state exception above.

### Flow 5: Content Deduplication (Multiple Events, Same Hash)

```
1. Event A with hash H: RC(H) = 1, A's content = Missing
2. Event B with hash H: RC(H) = 2, B's content = Missing
3. Content arrives, process A: A side effects, A content processed
4. Process B (same content): B side effects, B content processed
5. Delete A's content: RC(H) = 1, content still available for B
6. Delete B's content: RC(H) = 0, content can be GC'd
```

### Flow 6: Invalid Content

```
1. Event T arrives:
   - RC = 1
   - T's content = Missing

2. Content arrives, process_event_content_tx:
   - Side effects processing fails (e.g. CBOR deserialization error)
   - Set T's content = Invalid in `events_content_state`
   - Decrement RC (now 0)
   - Content bytes NOT stored in `content_store`

3. If author later deletes T's content:
   - old_state = Invalid
   - Set T's content = Deleted (records deletion intent)
   - RC NOT decremented again (already decremented)
```

## Idempotency Guarantee

The `Missing` state ensures content processing is idempotent:

```rust
fn can_insert_event_content_tx(...) -> bool {
    match events_content_state.get(event_id) {
        Some(Missing) => true,       // Process it
        None => false,               // Already processed
        Some(Deleted|Pruned|Invalid) => false, // Unwanted/bad
    }
}
```

This prevents duplicate side effects when:
- Same event is delivered multiple times
- Same content is delivered multiple times for same event

## Edge Cases and Guards

### Double-Decrement Prevention

When deleting/pruning content, we check old_state:

```rust
if !matches!(old_state, Some(Deleted { .. } | Pruned | Invalid)) {
    decrement_rc(...);  // Only if not already decremented
}
```

### Content Delete After Prune/Invalid

- Event content pruned/marked invalid: state = Pruned/Invalid, RC decremented
- Content deletion event arrives: state changes to Deleted, RC NOT decremented again
- Semantic: Author's content deletion intent is recorded, but no double-decrement

### Prune After Content Delete/Invalid

- Event content deleted/marked invalid: state = Deleted/Invalid, RC decremented
- Prune attempted: returns false (content already deleted/invalid)

### Event Already Present

```rust
if events_table.get(&event_id)?.is_some() {
    return Ok(InsertEventOutcome::AlreadyPresent);
}
```

Duplicate event delivery is a no-op. RC not incremented again.

## Content Fetch Scheduling

Missing content is fetched by the `MissingEventContentFetcher` task using an
event-driven approach with exponential backoff.

### Table Structure

The `events_content_missing` table uses a composite key `(Timestamp,
ShortEventId)` where the `Timestamp` is the scheduled next fetch attempt time.
This makes the table naturally sorted by when content should next be fetched.

### Missing State Metadata

The `EventContentState::Missing` variant tracks fetch attempt metadata:

```rust
Missing {
    last_fetch_attempt: Option<Timestamp>,  // when we last tried (fact)
    fetch_attempt_count: u16,               // how many times we tried (fact)
    next_fetch_attempt: Timestamp,          // when to try next (scheduling)
}
```

The `next_fetch_attempt` field mirrors the `Timestamp` component of the current
`events_content_missing` key, enabling removal (which requires the full
composite key). In canonical state, each event has at most one queue row. A row
is current only when the event is `Missing` and its timestamp exactly equals
`next_fetch_attempt`; non-Missing events have no current queue row. A Missing
event whose bytes are already available locally can temporarily have no queue
row while local processing completes. Legacy inconsistent physical rows can
remain behind valid work until lazy front repair or total replay, but queue
APIs filter them from current work.

### Fetcher Loop

Instead of scanning the entire missing table on a fixed interval, the fetcher:

1. Transactionally deletes inconsistent front rows until it finds an exact
   state/schedule match
2. Peeks at that first valid entry (smallest key = earliest due)
3. If due now: attempts to fetch from peers
4. If not due: sleeps until the scheduled time
5. If no valid row remains: waits for a `Notify` signal

A `Notify` channel wakes the fetcher immediately when new missing content is
inserted (via `on_commit` hook in `process_event_tx`).

### Backoff Formula

On fetch failure, the next attempt is scheduled with exponential backoff:

```
backoff_secs = min(60 * 1.5^(attempt_count - 1), 86400)
```

- Initial backoff: 60 seconds (1 minute)
- Maximum backoff: 86400 seconds (24 hours)
- New entries start with `next_fetch_attempt = Timestamp::ZERO` (try immediately)

### Failed Fetch Recording

The `record_failed_content_fetch` DB method:

1. Reads the current `Missing` state and compares its schedule with the
   caller-observed schedule
2. Ignores the completion if the state is no longer Missing or its schedule has
   changed, or if `next_attempt_at` is not strictly later than the current
   schedule
3. Removes the schedule entry mirrored by the current state
4. Inserts one schedule entry with updated `next_attempt_at`
5. Updates `events_content_state` with incremented count and timestamps

The caller provides both `attempted_at` (fact) and `next_attempt_at`
(scheduling decision). The backoff calculation lives in the fetcher, not the
DB layer. This compare-and-set behavior makes overlapping fetch completions
safe: only the completion for the current schedule can advance retry metadata.
Strictly increasing schedules prevent a duplicate completion from succeeding
through reuse of the same schedule value.

Queue peeking repairs legacy or otherwise inconsistent front rows in the same
write transaction used to select work. It removes rows for non-Missing or
absent events and rows whose timestamp differs from `next_fetch_attempt`, then
continues to later work without returning an empty result. A total migration
discards old queue rows and derives the current queue from retained events and
content; there is no separate queue-repair migration.

Queue pagination omits inconsistent rows even when they remain behind a valid
front row awaiting lazy repair, so diagnostic/API consumers see only current
fetch work.

## Potential Concerns

### 1. No Automatic Garbage Collection

When RC reaches 0, content remains in `content_store`. A separate GC process
should periodically clean up content with RC=0. (Future work)

### 2. Missing Events / Missing RC

These abnormal internal conditions are detected and logged:

- **Calling `process_event_content_tx` for a non-existent event**:
  `debug_assert!` + `error!` log, then silently skipped in release mode. The
  public `process_event_content` boundary inserts the carried envelope first.
- **Decrementing RC with no RC entry**: `debug_assert!` + `error!` log, then
  defaults to 1 to avoid underflow.

Both cases indicate bugs in the calling code and will panic in debug builds.

## Test Coverage

### Core Flow Tests

- `test_event_arrives_before_content` - Event before content flow
- `test_content_exists_when_event_arrives` - Content before event flow
- `public_content_ingestion_matches_combined_for_both_arrival_orders` - Public
  content-only, envelope-first, and combined ingestion converge across duplicate
  calls, reopen, and total replay, including Processed state, RC, queue, usage,
  and projections
- `public_content_ingestion_preserves_terminal_states` - Repeated public content
  ingestion preserves Deleted, Pruned, and Invalid lifecycle outcomes
- `test_multiple_events_share_content` - Deduplication + pruning
- `test_multiple_events_waiting_for_content` - Multiple events, same hash
- `test_delete_event_arrives_before_target` - Delete before target
- `test_predeleted_envelope_bookkeeping_converges` - Delete-before-target and
  target-before-delete produce equal self-envelope indexing, payload usage, RC,
  queue, and Deleted state across reopen and total replay
- `deleted_intermediate_lineage_converges_for_all_valid_deliveries` - All 90
  envelope-before-own-payload schedules produce identical transitive edit
  lineage, lifecycle bookkeeping, and semantic visibility
- `supplied_predeleted_edit_changes_only_lineage` and
  `supplied_predeleted_blank_delete_changes_nothing` - Supplied Deleted content
  adds only eligible nonblank edit metadata; blank deletion content is inert
- `retained_deleted_edit_lineage_survives_reopen_gc_and_total_replay` -
  hash-deduplicated retained content derives lineage without payload delivery,
  and immutable forward metadata survives reopen, byte removal, and total replay
- `over_limit_deleted_edit_derives_no_lineage_from_retained_or_supplied_bytes`
  - Both Deleted-content activation paths reject over-limit replacement content
- `test_content_processing_idempotency` - Duplicate content delivery
- `deleting_post_projection_reversion_is_symmetric` - Absent-body, empty, and
  whitespace deleting posts leave zero or unrelated nonzero reply/reaction
  state unchanged, while nonblank edits apply and revert authored-time, reply,
  news, and mention projections across target-first/delete-first delivery,
  duplicates, reopen, and total replay
- `ordinary_reaction_projection_still_reverts` - Eligible reaction projection
  insertion and checked deletion reversion remain active
- `delete_flag_without_aux_is_projection_inert` - A signed delete-aux flag
  without an auxiliary parent remains inert even with nonblank projection fields
- `inconsistent_eligible_reaction_reversion_fails_and_rolls_back` - An eligible
  projection with a corrupt zero aggregate fails checked reversion and rolls
  back deletion lifecycle and index changes, including both receipt directions
- `social_receipt_reversion_updates_cursors_without_reusing_order` - Reverting
  the latest ordinary post removes both receipt directions, moves raw latest and
  pagination cursors to the retained post, and preserves allocator durability
  and collision safety across duplicate deletion and reopen
- `inconsistent_social_receipt_mapping_aborts_complete_deletion` - A reverse key
  whose forward row is absent or resolves to another event fails closed and
  rolls back the complete deletion transaction
- `duplicate_social_receipt_mapping_aborts_complete_insertion` - A preexisting
  reverse mapping fails closed and rolls back the event, projections, and
  allocator while preserving the existing mapping
- `version_24_unmapped_receipt_is_rebuilt_before_open` - The version-25 total
  rebuild replaces an unmapped legacy receipt with exact authored-time forward
  and reverse membership before normal access
- `social_post_materialization_tests` - Atomic append/rollback, late payload
  materialization, exclusion paths, deletion/replacement resolution, bounded
  cursor and crash replay, enablement-tip baseline and concurrent boundary,
  sequence exhaustion, no-backfill cutover, exact rebuild preservation/retry,
  and fail-closed gap/lifecycle/content corruption

### Edge Case Tests

- `test_delete_while_unprocessed` - Content delete arrives while content is Missing
- `test_two_deletes_same_target` - Second content delete doesn't double-decrement RC
- `test_deletion_is_monotone_across_delivery_permutations` - Ordinary children
  cannot erase staged deletion
- `test_direct_deleters_converge_across_delivery_permutations` - Direct
  attribution uses canonical timestamp/ID precedence
- `test_equal_timestamp_deleters_use_event_id_tiebreak` - Equal-time direct
  attribution is total
- `test_deleter_attribution_update_does_not_repeat_reversion` - Canonical
  attribution updates do not revert processed projections twice
- `test_deletion_chain_preserves_header_effects` - Deletion chains retain direct
  non-transitive attribution, including staged ordinary-child interference
- `test_prune_then_delete` - Content Prune→Delete transition, no double-decrement
- `test_delete_then_prune` - Prune after content delete returns false
- `test_cross_author_parent_never_resolves_or_deletes` - Cross-author raw-ID
  matches stay author-scoped missing in either arrival order
- `test_process_content_for_nonexistent_event` - Silent skip (release only)
- `test_data_usage_payload_invalid` - Invalid content: Missing→Invalid, RC decremented
- `test_data_usage_invalid_payload_deletion` - Deleting invalid: Invalid→Deleted, no RC change
- `test_equal_timestamp_follow_conflicts_converge` - Follow/unfollow and selector
  conflicts converge in both orders
- `follow_epochs_converge_across_zero_one_and_two_unfollows` - Shuffled follow
  histories converge on active selector, current-epoch `first_ts`, and
  historical notification cutoffs
- `equal_second_event_order_defines_epoch_membership` - Same-second follow and
  unfollow events use the event-ID tie-break while the timestamp cutoff treats
  content at `first_ts` as current
- `post_and_shout_notifications_use_current_epoch_cutoff` - Both receipt indexes
  apply the current follow epoch's strict timestamp cutoff
- `metadata_only_epoch_changes_publish_followee_state` - Late nonwinning
  follow/unfollow changes publish the retained `first_ts` projection
- `follow_epoch_survives_reopen_and_total_replay` - Retained state and canonical
  replay derive the same follow epoch
- `test_equal_timestamp_latest_values_converge` - Profile and generic singleton
  conflicts converge in both orders
- `test_equal_timestamp_vote_conflicts_converge` - Vote winner and aggregate use
  one equal-second comparison
- `social_vote_winner_and_sum_update_atomically` - An aborted replacement
  transaction rolls back its event, winner, and aggregate together
- `colliding_full_vote_targets_converge_and_replay` - Different target authors
  sharing one shortened post ID transfer the winner contribution
  deterministically across delivery order and total replay
- `malformed_vote_shape_does_not_poison_projection` - A mismatched vote
  header/payload relationship cannot enter or block the vote projection
- `invalid_cached_vote_projection_fails_closed` - A vote row with a missing
  projection or a cached target outside its shortened key aborts reads and
  replacement without partial mutation
- `vote_winner_survives_deleted_and_collected_source` - Vote reads and
  replacements use the inline winner value after delete and source-byte
  collection
- `latest_singleton_query_is_isolated_ordered_and_strict` - Public singleton
  enumeration is identity/kind isolated, deterministically newest-first, and
  rejects malformed stored keys
- `test_failed_fetch_completions_are_compare_and_set` - Overlapping failed
  completions cannot leave stale rows after processing or deletion
- `test_failed_fetch_rejects_non_forward_schedule` - Equal or backward
  replacement schedules cannot reuse a current CAS token
- `test_missing_content_peek_repairs_stale_front_rows` - Fetcher peeking removes
  inconsistent front rows, pagination filters stale tail rows, later valid work
  remains reachable, and repairs persist across reopen
- `test_total_migration` - Total replay discards inconsistent queue rows and
  reconstructs canonical Missing state while preserving stable metadata,
  eligible referenced available content, extension rows, and event acquisition
  sources
- `total_migration_preserves_legacy_receipt_sources` - Authentic
  decode-incompatible version-6/11 and version-12 receipt layouts migrate while
  preserving local and populated network acquisition sources
- `version_25_adopts_pending_version_24_legacy_content_stash` - Version 25
  adopts a production-v24 legacy-content stash without replacing its source
  discriminator; a failed replay retains bytes for a successful retry
- `test_two_phase_replay_converges_across_event_orders` - Envelope-first replay
  converges across forward and reverse envelope/content scans without graph
  topology; receipt allocator values are intentionally excluded
- `total_replay_collision_fails_deterministically_and_remains_retryable` -
  Identity collisions abort replay without consuming the stash, and an
  identical reopen retries deterministically
- `total_replay_removes_legacy_exact_limit_deleted_edit_lineage` - Final replay
  removes exact-limit replacement lineage under the exclusive payload boundary
- `benchmark_large_total_migration_streaming` (ignored) - Manually rebuilds a
  10,000-envelope chain and reports elapsed time and file growth; run with
  `RUST_LOG=error cargo test -p rostra-client-db
  benchmark_large_total_migration_streaming -- --ignored --nocapture`

### Property and Shuffled-Order Tests

- [`property-testing.md`]property-testing.md documents the shared two-replica
  schedule runner, semantic models, exclusions, runtime budget, and soak command
- `prop_author_scoped_event_graph_converges` - Envelope graph indexes and
  unresolved canonical deletion attribution converge
- `prop_live_raw_content_lifecycle_converges` - Live RAW content bytes, RC,
  queue termination, and complete usage accounting converge
- `prop_terminal_content_lifecycle_converges` - Generated shared payloads,
  direct deletion, explicit pruning, eligible zero-RC byte collection, and
  usage buckets converge under one semantic oracle that ignores permitted
  physical residue
- `prop_replacement_projection_reversion_converges` - Deleting-post chains
  deterministically cover absent, empty, and whitespace bodies in both chain
  positions plus live and finally deleted edit chains, composing immutable
  replacement lineage with authored-time, reply, both public news orders,
  self-mention, receipt-membership, and visibility reversion
- `prop_follow_semantics_converge` - Follow winners, reverse membership,
  canonical epochs, and retained unfollow boundaries converge
- `prop_profile_and_singleton_semantics_converge` - Profile fields and
  profile/generic singleton winners converge
- `prop_vote_semantics_converge` - Per-voter shortened-key winners, full-target
  reads, and normalized numerical aggregates converge, including target-author
  collisions
- `test_shuffled_singleton_events_converge` - Shuffled finite latest-event sets
  select the total-order maximum

## Summary

The content lifecycle model handles:

- Out-of-order delivery (event before content, content delete before target)
- Duplicate delivery (idempotent via Missing state)
- Content deduplication (RC tracks multiple events per hash)
- Empty content (processed immediately at insertion unless already Deleted)
- Invalid content (failed validation, RC decremented, bytes discarded)
- Content deletion and pruning (with double-decrement prevention)
- Fetch scheduling (exponential backoff for missing content, event-driven wake-up)

The `Missing` state is the key to idempotency - it ensures content side
effects are applied exactly once per event, regardless of how many times the
content is delivered.