1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
//! Postgres-backed [`SendEventStore`].
//!
//! # Status
//!
//! Implemented, and run rather than asserted: **107 of 107** gated tests pass
//! against a live PostgreSQL 17.10 — the 95 rules of `event_store_conformance!`,
//! the 5 of `event_store_concurrency_conformance!` at `CONTENDERS = 64`, the
//! model family, and this crate's own six.
//!
//! One of those 105 is a **stated declension rather than a run**, and it is named
//! here rather than left to be discovered in a log: the fixture declines
//! `READ_YOUR_OWN_WRITES`, so `ops_agree_with_the_model` reports a skip carrying
//! the reason. The model family predicts whether a conditional append will be
//! rejected from what a read showed it, and on this store those are deliberately
//! different sets — see *What this store costs a caller* on the crate root. The
//! counts above read 101 and 89 until the `0.2.0` pass. The family was 93 rules
//! and had been since before this crate cleared it, so both numbers were
//! arithmetic on a stale one; it is 95 now, because ES-27 gained the two rules
//! that check a condition item's tags are AND-ed rather than keyed on the first.
//!
//! The concurrency family is the one that matters most: it is the first time an
//! adapter in this portfolio has cleared it against a store whose writers are
//! **not** serialised, which is the whole reason this crate is in the tree.
//!
//! The projection store beside this one is implemented too, and the crate carries
//! no `#![allow(clippy::todo)]` any more — it left with the last stub, which is
//! the contract it was written under.
//!
//! # Cancellation
//!
//! **A dropped `append` future cannot be cancelled by this adapter, and the
//! batch may already have committed.** ES-23 is `[FROZEN]` and obliges every
//! adapter to say which of its two outcomes it has; this is ours, and it is the
//! less comfortable one.
//!
//! The mechanism is `on_runtime`, which is `Handle::spawn(work).await`. Dropping
//! the caller's future drops only the `JoinHandle`. Tokio **detaches** a spawned
//! task rather than cancelling it, so the transaction underneath carries on: the
//! `INSERT` runs, the `COMMIT` runs, and there is nobody left to tell. Nothing
//! in this adapter observes the drop, so nothing can react to it.
//!
//! A caller **MUST NOT** read a dropped `append` future as evidence either way —
//! not that it committed, and not that it did not. The resolution is ES-24's:
//! reissue the identical batch. Event identity makes the reissue idempotent, so
//! the second attempt either lands (the first did not commit) or is refused as a
//! duplicate (it did), and both answers are true ones.
//!
//! This is precisely the shape ES-23's own `Rejects:` names, reached through
//! `spawn` rather than `spawn_blocking`. It is **not** `happenstance-sqlite`'s
//! answer and must not be described in its words: that adapter says there is
//! nothing to drop, because its `append` has no suspension point at all. Ours
//! has one, and what is on the other side of it keeps running.
//!
//! # The schema
//!
//! Authoritative in `migrations/0001_event_log.sql`, compiled in by
//! [`crate::migration::MIGRATION_1`] and applied by [`crate::migration::apply`].
//! It is not restated here: the SQLite adapter keeps its schema as a Rust
//! constant mirrored into a doc comment and needs a test comparing the two
//! against `sqlite_master`, because the copies drift. A file has no second copy
//! to drift from.
//!
//! What is worth reading before the file: `position` carries **no column
//! default** and is not `bigserial`; the sequence is read explicitly at the one
//! `INSERT` that allocates; and each row stamps `pg_current_xact_id()` into an
//! `xid8` column, which is the mechanism every read and `head` filter on.
//!
//! `position` is deliberately **not** `bigserial`. That is the whole subject of
//! this module's ES-10 note below: a `serial` column is `nextval()`, and
//! `nextval()` is where the invariant is lost.
//!
//! # ES-10, and what this adapter pays for it
//!
//! ES-10 says position order is visibility order: once a reader has seen
//! position *P*, nothing at or below *P* may appear later. A naive Postgres
//! adapter breaks it, because `nextval()` allocates outside the transaction — a
//! writer takes 99, a writer that started later takes 100 and commits first, and
//! a reader who conditioned on `after: Some(100)` never sees the 99 that would
//! have violated its boundary. There is no error, no rejected append and no
//! failing test; there is a read model that is correct about a state the
//! business forbids.
//!
//! **This adapter buys the invariant with `xid8` + `pg_snapshot_xmin`, and the
//! choice was measured rather than preferred.** ADR-0024 is the record. Each row
//! stamps `pg_current_xact_id()`; every read and `head` admit only rows below
//! `pg_snapshot_xmin(pg_current_snapshot())`, the frontier beneath which no
//! transaction can still be in flight. The mechanism does not deny that
//! `nextval()` allocates outside the transaction — it stops treating position
//! order as *visibility* order and moves the guard to the read side.
//!
//! What that asks of this adapter's types and its append path:
//!
//! * **Writers do not serialise at all**, which is the property this crate
//! exists to have, and [`SendEventStore::append`]'s signature is untouched.
//! What the async boundary did force is the conditional path: under
//! `READ COMMITTED` two writers racing one boundary both find no conflict and
//! both win, so a conditional append runs `SERIALIZABLE` — whose SSI is
//! optimistic and takes no locks, so disjoint boundaries still proceed in
//! parallel — and retries a `40001`. Holding a write lock instead is the move
//! this crate exists *not* to make.
//! * **The cost is read-side, and it is structural rather than incremental.**
//! Every read gains the visibility predicate, and the predicate is composed
//! into the cursor's `DECLARE` inside one `REPEATABLE READ` transaction and
//! evaluated once. Not per `FETCH`: the frontier advances between chunks, and
//! a per-chunk predicate lets rows appear beneath positions the caller has
//! already been handed — ES-10's own violation arriving through the read path
//! while the write path was being fixed.
//! * **`head` reports the frontier rather than the maximum position**, so
//! freshly committed events are invisible until the frontier passes them. That
//! is a **capability** limit rather than a type error: the adapter compiles,
//! and then read-your-own-writes does not hold.
//!
//! **The two numbers.** Steady state, measured against this built adapter with
//! the predicate removed as the paired baseline: the median ratio straddles 1.0
//! at every concurrency level with about a 20% spread, so no cost large enough
//! to matter is measurable — and the arms that cost 16x and 30x would be
//! unmissable at that precision. Staleness, which is the real bill: a median of
//! 0.593 ms unloaded and **4,799 ms behind a five-second write transaction held
//! anywhere on the cluster**, against an unguarded arm unaffected by the same
//! hold at 0.595 ms.
//!
//! **The alternative that lost, named once so it is not re-proposed.** A
//! tag-keyed advisory lock is the cheapest rival — 0.935 of baseline at 64
//! writers, nearly free — and it lost on the **invariant** rather than on cost:
//! it buys a per-boundary property where ES-10 states a global one, which makes
//! `AppendCondition` sound and the projection checkpoint unsound. Ranking the
//! arms by throughput would have selected it. The two that serialise every
//! writer, a sequence table and a constant advisory lock, are correct and lost
//! on cost at 0.062 and 0.033.
//!
//! What follows for a caller, and the crate does not soften it: `head` is a
//! frontier, read-your-own-writes does not hold and is not claimed, and the
//! staleness bound is the longest open write transaction on the cluster — which
//! a consumer neither controls nor can necessarily observe. The crate root
//! states that bill where a consumer meets it first.
//!
//! `references/adr/0024-position-visibility-mechanism.md` carries the argument,
//! the losing arms and what each lost on, and the parts this decision inherits
//! rather than owns.
use Stream;
use ;
use ;
use Future;
use ;
use ;
use Duration;
use Handle;
use cratePostgresEventStoreError;
use cratePgReadStream;
/// A Postgres-backed event store.
///
/// Holds a [`PgPool`] rather than a connection, which is the shape difference
/// that makes this crate an instrument: readers and writers do not queue behind
/// one another, so nothing about the storage layer supplies ES-10 for free.
///
/// # What this store does not promise
///
/// It buys ES-10 on the read side, with `xid8` + `pg_snapshot_xmin`, so
/// [`head`](SendEventStore::head) reports a **visibility frontier** rather than
/// the highest position assigned, and **read-your-own-writes does not hold**.
/// Staleness is bounded by the longest open write transaction anywhere on the
/// cluster. Those are capability limits rather than defects; the crate root
/// states them in full, and [`append`](SendEventStore::append) and
/// [`head`](SendEventStore::head) each state the half a caller meets there.
/// Whether a store admits rows its transaction cannot yet vouch for.
///
/// Two arms of one adapter rather than two adapters, which is the point: the
/// naive arm differs from the shipped one in exactly the predicate under test
/// and in nothing else, so a rule that fails against it is failing on the
/// mechanism rather than on an unrelated difference between two codebases.
pub
// `SendEventStore`, not `EventStore`. A `PgPool` is `Send + Sync` and every
// future below crosses threads happily; implementing the `Send` flavour gives
// the bare one for free, and the implication runs only in that direction.
/// How many times an append re-runs after a serialisation failure.
///
/// A conditional append runs `SERIALIZABLE`, so two writers racing the same
/// boundary produce a `40001` for one of them rather than two winners. The retry
/// is not a workaround for that: on the second attempt the loser *sees* the
/// winner's committed row and returns `ConditionViolated`, which is the answer
/// the caller was owed all along.
///
/// **This was three, and three is measurably not enough.** The justification it
/// carried — *"the retry is only ever needed once in the two-writer case"* — is
/// true of the two-writer case and says nothing about the case that actually
/// exhausts the budget. `happenstance-neon` re-derived the number against a live
/// endpoint and wrote down why — `NeonEventStore::SERIALISATION_ATTEMPTS` carries
/// the table, and it is deliberately named in prose rather than linked, because
/// `happenstance-neon` is not a dependency of this crate and no adapter may
/// depend on another:
/// `k_disjoint_boundaries_admit_exactly_k_commits` runs twelve contenders over
/// **four separate boundaries**, and the SSI predicate lock is *not* per
/// boundary — with a small table the planner takes a sequential scan and the
/// lock is relation-wide, so every contender conflicts with every other whatever
/// boundary it is racing. A retry that finds no conflict on its own boundary goes
/// straight back into the same fight, and the population drains one round at a
/// time instead of all at once. Measured there, four runs per value: **three
/// failed 2 of 4**, four and five were green 4 of 4, eight green 5 of 5.
///
/// **That cause belongs to Postgres and not to Neon** — Neon *is* Postgres over
/// one-shot HTTP, and the predicate lock is the server's. This adapter kept three
/// because nothing had re-measured it, and CI found it the first time the live
/// suite ran on a pull request: two contenders exhausted the budget and surfaced
/// `Failed("postgres rejected the work")` where the rule requires a commit or a
/// `ConditionViolated`.
///
/// **Eight alone did not fix it, and the reason is the whole finding.** Raised
/// from three, the rule failed again on CI with *more* exhausted contenders —
/// two at three attempts, five at eight. A budget is not a mechanism: without a
/// wait between them, every loser re-enters at once and collides with every other
/// loser, so extra attempts are extra transactions in the same fight. What was
/// missing is spacing, and [`backoff`] is where it now lives — read that first,
/// because this constant only bounds a loop that behaves the way that function
/// makes it behave.
///
/// **The number is adopted rather than re-measured here, and that is stated
/// rather than smoothed over.** Eight comes from Neon's table against a shared
/// cause; it is not four runs per value against a live PostgreSQL server, which
/// is the bar ADR-0024 sets for a choice like this. Re-measuring both this and
/// the backoff window together is owed, and should be done on a CI-sized runner
/// rather than a developer machine — local hardware is precisely what hid this.
///
/// It still costs nothing when it is not needed: a retry happens only after a
/// `40001`. When it is needed the cost is now eight round trips *plus* the waits,
/// which [`BACKOFF_CAP`] bounds at roughly a quarter of a second in the worst
/// case, paid only by a writer already losing a serialisation fight.
const SERIALISATION_ATTEMPTS: u32 = 8;
/// Postgres's `serialization_failure`.
const SERIALIZATION_FAILURE: &str = "40001";
/// An append's outcome, before it is mapped onto the port's error type.
/// Runs one append: condition and insert on one snapshot, in one transaction.
///
/// # Why the isolation level depends on whether there is a condition
///
/// An unconditional append asserts nothing about the log's state, so it cannot
/// conflict with anything. It runs at the pool's default `READ COMMITTED` and
/// pays nothing.
///
/// A conditional append is a read and a write over the same predicate, which is
/// exactly the shape `READ COMMITTED` cannot make safe: two writers both find no
/// conflicting event, both insert, and both win. The other adapters in this
/// workspace avoid that by holding a write lock — `BEGIN IMMEDIATE` on SQLite —
/// which is precisely the move this crate exists *not* to make, because a store
/// that serialises every writer is `MemoryEventStore` with network latency.
///
/// `SERIALIZABLE` is the arm that buys the invariant without the lock. Postgres's
/// SSI is optimistic: it takes no locks and detects read-write dependency cycles
/// at commit, so two writers on **disjoint** boundaries never conflict and the
/// axis this crate occupies survives. Two writers on the **same** boundary do
/// conflict, and one of them is aborted — which is the elected loser.
///
/// This is a shape decision the spec left open (`_decomposition.md` Architecture
/// §9.3: whether Postgres reuses Neon's single-statement CTE or exploits the
/// interactive transaction it has and Neon does not). It is recorded here rather
/// than defaulted: the interactive transaction is exploited, because a CTE cannot
/// express "retry after a serialisation failure" and Neon's CTE exists to work
/// around the absence of a transaction rather than because it is better.
async
/// The shortest wait between two attempts, doubled per attempt from here.
const BACKOFF_BASE: Duration = from_millis;
/// The longest wait between two attempts, whatever the exponent says.
///
/// Sixty-four milliseconds is a ceiling rather than a target: with full jitter
/// the *expected* wait is half the window, and `SERIALISATION_ATTEMPTS`
/// exhausting at this cap costs a caller roughly a quarter of a second in the
/// worst case — paid only by a writer already losing a serialisation fight.
const BACKOFF_CAP: Duration = from_millis;
/// How long to wait before re-running an aborted conditional append.
///
/// # Why a retry needs a wait at all, which this adapter did not know until CI
///
/// `SERIALISATION_ATTEMPTS`'s doc block explains why *more* attempts were needed
/// and it is only half the story; the other half is that attempts have to be
/// **spread out**, and nothing here was spreading them.
///
/// `k_disjoint_boundaries_admit_exactly_k_commits` races twelve contenders over
/// four boundaries. The SSI predicate lock is relation-wide on a small table, so
/// all twelve conflict with each other whatever boundary they are racing. With no
/// wait, every loser re-enters at once and collides with every other loser — a
/// thundering herd that a bigger budget makes *worse* rather than better, because
/// each extra attempt is another transaction in the same fight. Measured on CI:
/// the rule failed with two exhausted contenders at three attempts and **five** at
/// eight. The budget was the wrong lever.
///
/// **`happenstance-neon` does not need this and that is why the number did not
/// transfer.** Its retry costs a network round trip to a pooled proxy — on the
/// order of a hundred milliseconds — so its transport supplies the spacing for
/// free, and its doc block says so in terms: *"the retry itself is the backoff"*.
/// That sentence is true there and false here, where a retry against a pooled
/// local connection costs well under a millisecond. Adopting Neon's value without
/// its transport adopted half a mechanism.
///
/// # The shape: exponential, capped, full jitter
///
/// Full jitter — uniform in `[0, ceiling]` rather than `ceiling ± a bit` — is the
/// arm that actually decorrelates a herd. Equal-and-opposite jitter around a
/// common centre leaves the population clustered at that centre, which is the
/// defect being fixed.
///
/// The randomness is [`RandomState`] rather than a `rand` dependency. Each
/// `RandomState::new()` carries a fresh seed, so hashing a constant yields an
/// independent draw per call with no new licence surface, no advisory surface and
/// no MSRV exposure — which the workspace manifest is explicit about wanting for
/// a crate that needs exactly one thing.
///
/// # What it requires of the caller's runtime, and why that is not new
///
/// [`tokio::time::sleep`] needs a runtime with the timer enabled. This adds no
/// requirement: `sqlx`'s pool already enforces an acquire timeout, so a runtime
/// that could not tell the time could never have handed this store a connection.
/// A uniform draw in `[0, 1]`, from the standard library and nothing else.
///
/// `RandomState::new()` is freshly seeded per instance, so the hash of a constant
/// differs between calls. The top thirty-two bits are taken because `f64::from`
/// is lossless on `u32` and a `u64`-to-`f64` cast is not — the same reason the
/// crate's lints would refuse the shorter spelling.
/// True for Postgres's `40001`, and only for it.
///
/// Matched on the `SQLSTATE` rather than on the message, and deliberately not
/// turned into an error variant: `sqlx` already exposes the code, and a variant
/// per `SQLSTATE` is the taxonomy this crate's `error.rs` explicitly refuses.
/// One attempt: begin, evaluate, insert, commit.
async
/// Evaluates every guard, returning the highest conflicting position if any is
/// violated.
///
/// A guard is violated when its query matches an event at a position **strictly
/// greater** than `after`, or at any position when `after` is `None`. Any guard
/// violated violates the condition.
///
/// # No frontier predicate here, deliberately
///
/// `head` and `read` carry `xact_id < pg_snapshot_xmin(…)`; this does not. The
/// frontier exists so a *reader* never sees a position appear beneath one it has
/// already observed. A condition is not a read: it asks whether the store holds
/// a conflicting event at all, and a committed row above the frontier is held.
/// Filtering it out would admit an append the boundary forbids — the same silent
/// corruption this whole mechanism exists to prevent, arriving through the door
/// left open while the other one was being closed.
async
/// Inserts the whole batch as one statement and returns the last position.
///
/// # Why positions are allocated in their own statement
///
/// `nextval('event_position_seq')` is read explicitly rather than as a column
/// default, so the allocation site is visible at the one statement that performs
/// it. Taking the values up front also lets the `origin_position` half of each
/// `EventId` be bound directly, instead of being stamped by a follow-up `UPDATE`
/// the way the SQLite adapter has to.
///
/// # Why one statement rather than a row at a time
///
/// The batch ceiling is 256 events at eight columns — 2,048 bound parameters,
/// comfortably inside the extended protocol's `int16` limit of 65,535, and one
/// round trip instead of 256.
async
/// Decodes a stored `bigint` into a [`SequencePosition`].
///
/// `bigint` is signed and admits `0`; `SequencePosition` is a `NonZeroU64`. The
/// rejected alternative is `SequencePosition::new(stored.unsigned_abs())`, which
/// reads like a guard and accepts every negative value, turning a stored `-3`
/// into position 3.
pub
/// A position as the `bigint` the schema stores.
/// The wall clock, in the units [`RecordedAt`] carries.