1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
//! Whole-record hydration: the per-record COLLECTION PLAN, the two statements a dialect
//! materializes an edge-stored collection with, and the decode that makes an absent collection the
//! empty one.
//!
//! ```text
//! Postgres (SELECT array_agg(e.<value> ORDER BY e.<ordinal>) FROM <edge> e WHERE e.k = p.k)
//! rides the page's own SELECT list 1 statement, no tear
//!
//! SQLite SELECT <keys…>, <value> AS "__element", <ordinal> AS "__ordinal" FROM <edge>
//! MySQL WHERE (<keys…>) IN ((?, …), …) ORDER BY <keys…>, "__ordinal"
//! one statement per collection column, batched over the page 1 + N, one connection
//! ```
//!
//! # ELEMENTS TRAVEL TYPED CHANNELS ONLY — there is no JSON anywhere on this path
//!
//! A native array column or typed rows, and never a text encoding: not `json_group_array`, not
//! `JSON_ARRAYAGG`, not between the two statements. The edge tables exist because JSON was refused
//! as STORAGE, and the same three reasons refuse it as transport — a blob element cannot ride it,
//! MySQL's `JSON_ARRAYAGG` cannot be given an order, and it would put a codec layer between two
//! parties that already agree on a type. So every element below reaches Rust through the
//! [`Carrier`](crate::Carrier) table's own decode, exactly as a scalar column does.
//!
//! # The symmetry is the design
//!
//! The dialect that stores arrays natively aggregates them natively; the dialects that need an edge
//! table take the gather. One seam — the plan below — and two materializations, so a page is one
//! statement where the engine can make it one and a bounded batch where it cannot.
//!
//! Which of the two a column takes is the PLAN's and not the dialect's, and that is not a
//! generalization for its own sake: Postgres has both. An element its array can hold is an ordinary
//! column that rides the row and never appears here; an element it cannot — and any element whose
//! declaration named the edge strategy anyway — is aggregated. See
//! [`PgCollectionStorage`](crate::comparison::sql::PgCollectionStorage), where that per-element
//! choice is already made and where the same "may name the edge strategy anyway" escape is stated.
//!
//! # An absent collection is the EMPTY one, structurally
//!
//! A zero-row aggregate is SQL `NULL` and a parent with no edge rows is absent from the gather, and
//! neither is a case anything here tests for. A row arrives from `from_row` with its collections
//! EMPTY and materialization only ever ADDS, so the empty collection is what a row keeps when
//! nothing fills it. That is the collection collapse — *empty ≡ none* — held by the shape of the
//! decode rather than by a branch that could be written the other way.
//!
//! # No chunking, and the derivation is arithmetic rather than a hope
//!
//! The gather's membership binds one parameter per key column per PAGE ROW, and a page is clamped
//! to [`MAX_PAGE_SIZE`](crate::query::MAX_PAGE_SIZE) — 1000 — before any statement renders. Against
//! the ceilings [`BindCeiling`] states — 32766 on SQLite, 65535 on Postgres and MySQL — a key of up
//! to **32 columns** cannot reach the lowest of them, so the batch is never split and
//! [`SqlKeyIn`](crate::comparison::sql::SqlKeyIn)'s refuse-don't-chunk doctrine is never asked for.
//! The check below is still made, because a key WIDER than that is expressible and a silent
//! truncation there would drop elements rather than refuse.
//!
//! # What was MEASURED, and why the rendering tests beside this cannot see it
//!
//! Every form here renders plausible SQL when it is wrong. A gather that never runs, one correlated
//! to the wrong parent, one whose order lost the ordinal — each answers without an error and gives
//! back a different collection on a right row, which is the defect class the comparison family's own
//! edge module records one level down.
//!
//! So the check is a CORPUS rather than a text: six rows over two lineages, carrying BOTH shapes —
//! a sequence holding an empty collection, a single element, a REPEATED element and one whose edge
//! rows were written back to front so their physical order disagrees with their ordinals, and a set
//! whose members were written in an order that is not their value order.
//!
//! | engine | what was put to it | answer |
//! |---|---|---|
//! | **SQLite 3.51.0** | the whole engine — every lane, `get`, the snapshot, and the page walk two rows at a time in both directions, every cursor round-tripped through its base64 string, over a record carrying one collection of each shape | exact, every collection matching the one computed in Rust from the data |
//! | **PostgreSQL 14.23** | the page statement this module renders for the same record, both aggregates on it | exact — `{a,b,c}` for the sequence written `c, b, a`, `{dup,dup}` for the repeated one, `{alpha-note,beta-note}` for the set written the other way round, and SQL **`NULL`** rather than `'{}'` for every parent with no elements |
//! | **MySQL 8.4.11** | both gathers this module renders, whose two forms beyond what [`SqlKeyIn`](crate::comparison::sql::SqlKeyIn) already measured there are a select-list ALIAS ordered by and a bare `AS` projection of an edge column | exact, the sequence in ordinal order and the set in value order |
//!
//! Two mutations were put to the same corpus, and one to the compiler:
//!
//! | mutation | what came back |
//! |---|---|
//! | the gather never runs | every collection EMPTY, with every row and every scalar column right — four of the live tests red and every rendering test still green, which is the shape of the whole defect class |
//! | the gather's order drops the ordinal | nothing at all on SQLite, whose plan SEARCHes the edge table's own primary-key index — that key IS `(key…, ordinal)`, so its rows arrive in that order anyway. On **PostgreSQL** the same statement takes a bitmap heap scan, and the row written `c, b, a` came back `c, b, a`: the corpus's INSERTION order, silently, with no error |
//! | the gather takes a connection of its own | it does not compile, twice over — a second `acquire` on the executor is `E0382: use of moved value` (sqlx's `Acquire::acquire` takes `self`), and handing the gather an `Acquire` in place of the connection is `E0277: the trait bound &mut <Db as Database>::Connection: Acquire<'_> is not satisfied` |
//!
//! The second row is why the ordinal term is STATED rather than left to the index: on the engine
//! whose plan reads the heap it is the whole of the sequence, and on the engine whose plan reads the
//! index it costs nothing to say. It is also why the SQLite walk is not the whole of the check — an
//! engine that answers right for a reason the statement did not give is an engine that will stop.
//!
//! The `NULL` in the first table is why the aggregate's crossing decodes `Option<Vec<E>>`; see
//! [`aggregated`].
use HashMap;
use ;
use crate::;
/// The name every gathered element is projected under.
///
/// An ALIAS rather than the column's own name, so the party decoding one names what the ENGINE
/// wrote rather than re-deriving the column path the engine rendered — two spellings of one name,
/// free to drift. It carries the `__` the comparison family's own aliases do and cannot collide with
/// a declared column for the same reason theirs cannot.
pub const ELEMENT: &FieldName = new;
/// The alias one SEGMENT of a composite element is gathered under: [`ELEMENT`] and the segment's own
/// name, joined the way a column path is joined.
///
/// Public to this module for the reason [`ELEMENT`] is public to the crate — the party that DECODES
/// a gathered row reads the name the engine wrote, and a composite's row carries several. It is
/// spelled here so that the projection, the `ORDER BY` over a set's own key and the decode all read
/// ONE derivation, which is the naming contract this whole family keeps.
/// The name a sequence's ordinal is projected under, and the term its gather orders by.
///
/// Ordering by a SELECT-list alias rather than by the column expression is what keeps every term of
/// this statement's `ORDER BY` a plain NAME, which is the only thing
/// [`QueryBuilder::order_by`](crate::comparison::sql::QueryBuilder::order_by) renders — so the
/// gather needs no second escaping rule for an ordering position. All three dialects resolve an
/// output column's name there, and each still reads the underlying column for the index.
const ORDINAL: &FieldName = new;
/// The segment an edge table's ordinal column is one frame below the value column at — the same
/// derivation [`SqlEdgeComparison`](crate::comparison::sql::SqlEdgeComparison) makes, so
/// `<field>_ordinal` is one rule rather than two spellings of one convention.
const ORDINAL_SEGMENT: &FieldName = new;
/// The prefix an aggregated collection's SELECT-list alias is built from — `__element_0` for the
/// first collection of the plan, `__element_1` for the second.
///
/// Several aggregates ride ONE statement, so unlike the gather's single alias these have to be told
/// apart; the plan's own index is what tells them apart, and the engine hands the alias it wrote to
/// the decode rather than expecting it to be recomputed.
const AGGREGATE_ALIAS: &str = "__element_";
/// One collection column of a row, and how this database materializes it over a page.
///
/// # It is DECLARED rather than deduced, for the reason the strategy one level down is
///
/// Whether a column is a collection at all, what shape its edge table has, and which of the two
/// materializations this database takes are three facts about a DECLARATION. A `Vec<T>` behind an
/// alias is a scalar to anything reading tokens, so nothing can recover them by looking; the party
/// that wrote the declaration states them, which is exactly what
/// [`PgCollectionStorage`](crate::comparison::sql::PgCollectionStorage) already asks of an element
/// type.
///
/// # A column named here is NOT in [`SqlQuery::COLUMNS`]
///
/// That is the contract, and it is what makes both materializations one rule rather than two: a
/// gathered column has no value on the page row to select, and an aggregated one is an EXPRESSION
/// the engine renders rather than a column name. Selecting it as a bare name would ask the page's
/// own table for a column that is not in it.
///
/// # The typed halves are function pointers, and that is what keeps them typed
///
/// An element's type is the declaration's, so nothing generic over the dialect can name it. What
/// the engine owns is the STATEMENT — the edge table, the membership, the order, the connection and
/// the bind ceiling — and what a declaration owns is the crossing at each end. [`aggregated`] and
/// [`distribute`] are those crossings written ONCE, so a plan entry names the types and is not a
/// place where the grouping could be got wrong.
/// Bind one page row's key column, in the order [`SqlQuery::ADDRESS`] names them.
type GatherKey<Row, Db> = fn;
/// Give every page row the elements gathered for it.
type GatherTake<Row, Db> =
fn ;
/// One gather, ready to run: the statement, and the crossing that absorbs what it returns.
type Gather<Row, Db> = ;
/// Take one aggregated array off the page row it rode in on.
type AggregateTake<Row, Db> =
fn ;
/// The payload of [`Plan::Gather`]: the bind of one key column, and the crossing that absorbs what
/// the second statement returns.
///
/// It is `GatherPlan` and not `Gather`, which the ready-to-run pair above already holds — the two
/// are the plan and its product, so they are the one pair in this module that cannot share a name.
///
/// `Clone` and `Copy` are WRITTEN OUT for the reason [`Plan`]'s are.
/// Which of the two materializations a column takes, and the typed crossing it needs.
///
/// `Clone` and `Copy` are WRITTEN OUT rather than derived, for the reason every `Db`-parameterised
/// type in the comparison family writes its own: a derive would bound `Row` and `Db`, and this holds
/// a value of neither.
/// The elements a correlated aggregate put on the page row — a plan's
/// [`aggregate`](EdgeCollection::aggregate) crossing, written once.
///
/// # SQL `NULL` and the empty array are the same answer
///
/// `array_agg` over no rows is `NULL` rather than `'{}'` — measured on PostgreSQL 14.23, where the
/// corpus's empty-collection row answers `IS NULL` true and its neighbour false. So the two
/// spellings of *this parent has no elements* both arrive here and both leave as the empty [`Vec`],
/// and decoding into `Option<Vec<E>>` is what absorbs the fork.
///
/// What a decode into `Vec<E>` would do with that `NULL` is the DRIVER's business rather than a
/// fact about the data, and neither of its two answers is the right one: it may refuse, turning
/// *this parent has no elements* into *this query failed*, or it may hand back the type's default,
/// which is the silent kind. Both really happen — measured on sqlx 0.9, a `NULL` SQLite `DOUBLE`
/// read as `f64` rather than `Option<f64>` comes back `0.0` with no error at all. So the collapse
/// is stated here rather than left to whichever of the two a dialect's element type happens to
/// take.
///
/// # Errors
///
/// Whatever the driver refuses the decode with — an element that is not this column's type, or an
/// alias the statement did not project.
/// Give every page row the elements gathered for it — a plan's [`gather`](EdgeCollection::gather)
/// crossing, written once.
///
/// `key` reads a page row's own parent key, `parent` reads the one a gathered row carries, `element`
/// decodes the element it holds, and `into` names the collection it belongs in. The engine gathered
/// them in `(parent, ordinal)` order, so appending in arrival order IS the stored sequence.
///
/// # A parent with no gathered row keeps what it was decoded with
///
/// There is no backfill step and no `else` branch: a page row arrives with its collection empty and
/// this function only appends, so a parent absent from the gather ends with the empty collection —
/// which is the value a parent whose edge table holds nothing for it should have. The collapse is
/// the shape of the loop rather than a case in it.
///
/// # A gathered row whose parent is not on the page is IGNORED
///
/// It cannot arise from a statement this engine rendered — the membership is exactly the page's own
/// keys — so the only way to see one is a second writer inserting under a key this page does not
/// hold, and dropping it is what keeps the page the set of rows the page statement returned.
///
/// # Errors
///
/// Whatever the driver refuses a decode with, from either the parent key or the element.
/// The names one collection's second table is addressed by, each derived through an operation the
/// comparison family already states rather than re-joined here.
///
/// # Each is the BUFFER its renderer hands back, and none of them is a `String`
///
/// A name is HELD here rather than pushed as it is rendered, because a materialization spells
/// several of them more than once — the aggregate names the edge table three times, and its value
/// column twice over where the collection is a set. What holding one costs is the renderer's own
/// type and nothing on top: the two identifiers are [`Utf8BytesMut`], which is what [`quote_ident`]
/// returns, and the two columns are [`Backend::Buffer`](crate::Backend::Buffer), which is what
/// [`render_column`](crate::comparison::sql::render_column) returns. A name of ordinary length is
/// inside that buffer's inline range, so where four owned `String`s were four allocations
/// unconditionally, these take at most ONE — the correlation's, and only once it runs past 62 bytes.
///
/// The two halves are two TYPES where the correlated comparison kinds one level down keep three of
/// one: an identifier is this family's to bracket, a COLUMN is the backend's to render, and those
/// kinds hold both as one type only because they pin
/// [`Buffer`](crate::Backend::Buffer) — which is the equality
/// [`SqlDialect`](super::SqlDialect) deliberately does not state, so no query lane may assume it.
/// It is `pub(super)` because the WRITE lane derives the same three names — see
/// [`write`](super::write), whose edge statements name the same table, the same value column and the
/// same ordinal a materialization reads. One derivation with two readers, rather than a second
/// party joining the names again: the naming contract this module records lists its parties, and a
/// write that spelled its own would be a fifth.
pub
/// Derive them, walking the plan's path down from the row exactly as a lowering walks a comparison's.
///
/// # Errors
///
/// [`SqlLoweringError::EdgeWithoutField`] for the EMPTY path, which is
/// [`edge_table`](crate::comparison::sql::edge_table)'s own refusal: extended by nothing, the row's
/// table is its own, and every correlation built from it would join the row to itself.
pub
/// Push one aggregated collection onto `builder`'s SELECT list, under the alias its index names.
///
/// ```sql
/// (SELECT array_agg("t_tags"."tags" ORDER BY "t_tags"."tags_ordinal")
/// FROM "t_tags" WHERE "t_tags"."id" = "t"."id") AS "__element_0"
/// ```
///
/// A SET aggregates in its own primary key's order, which is `(key…, value)` — free, since that is
/// the index the correlation already seeks, and deterministic, which an unordered aggregate is not.
/// That is why the ordering term is the ordinal OR the value rather than a branch over two forms:
/// both shapes order by a column of the edge table, and only which column differs.
///
/// # The expression is written into the statement rather than built and then pushed
///
/// It is fourteen pushes of names this function already holds, and the `String` it used to be
/// existed only to carry them the one step from here to
/// [`push_field`](crate::comparison::sql::QueryBuilder::push_field), which takes a whole field at
/// once. [`push_field_with`](crate::comparison::sql::QueryBuilder::push_field_with) opens the field
/// instead, so the pieces land in the statement's own buffer and the separator stays the builder's.
///
/// The refusal comes FIRST, and that ordering is what keeps it clean: [`edge`] runs before any
/// push, so a plan entry naming no field leaves the SELECT list exactly as it was found rather than
/// half a field into one.
///
/// # Errors
///
/// As [`edge`].
/// The SELECT-list alias the `index`th collection of a plan is projected under.
///
/// It is held rather than pushed because the two parties that need it are a statement and a DECODE
/// — [`push_aggregate`] brackets it and [`absorb`] hands it to the driver as the column to read —
/// so there is no one buffer for it to be written into. A `Utf8BytesMut` is what makes holding it
/// free: `__element_` and a plan index are inside the inline range, where a `format!` allocated
/// once per collection per statement and once more per collection per ROW.
/// Push every aggregated collection of `Row`'s plan onto a statement's SELECT list.
///
/// It runs for EVERY lane, `get` included: a whole record is a whole record whichever verb asked
/// for it, and a page that hydrated where a single get did not would answer the same question two
/// ways.
///
/// # Errors
///
/// As [`edge`], carried verbatim — a plan naming a column the row has no edge table for is a
/// refusal about the declaration and not about the query.
pub
/// The statement one gathered collection takes over `page`.
///
/// ```sql
/// SELECT "media_id", "id", "tags" AS "__element", "tags_ordinal" AS "__ordinal"
/// FROM "clip_tags" WHERE ("media_id", "id") IN ((?, ?), (?, ?))
/// ORDER BY "media_id" ASC, "id" ASC, "__ordinal" ASC
/// ```
///
/// The membership is the row-value `IN` [`SqlKeyIn`](crate::comparison::sql::SqlKeyIn) renders, over
/// the page's own keys — one spelling in all three dialects, measured there. It is written here
/// rather than borrowed because a key set built from N ROWS has no single value type to hold: each
/// column binds through the plan's own `key`, at that column's own stored type.
///
/// The `ORDER BY` is the edge table's own primary key — `(key…, ordinal)` for a sequence and
/// `(key…, value)` for a set — so an engine that seeks that index has nothing to sort. One does and
/// one does not: SQLite SEARCHes it and PostgreSQL, on the corpus in the module note, took a bitmap
/// heap scan and a Sort. Which is exactly why the term is STATED rather than relied on. What it
/// buys is the SEQUENCE: a stored collection's order is the ordinal column and not the physical
/// one, and the two agree only until a row is rewritten.
///
/// # Errors
///
/// [`SqlLoweringError::BindCeiling`] where the page's keys cost more parameters than the dialect
/// binds — see the module note for why the arithmetic says a key of 32 columns or fewer cannot —
/// and [`edge`]'s refusal for a plan entry naming no field.
/// Every gather `Row`'s plan renders over `page`, each beside the crossing that absorbs it.
///
/// Split out from [`hydrate`] because the STATEMENT is what a rendering test can see and the fetch
/// is what it cannot — the same split the three lanes make one module over.
///
/// # Errors
///
/// As [`gather_statement`].
pub
/// Run every gather of `Row`'s plan on `conn`, and give the page its collections.
///
/// # ONE connection, and it is the one the page statement ran on
///
/// The connection arrives already acquired, so a gather cannot open a second one — there is no
/// executor here to acquire from. That is what keeps the cells the ruling calls tearing inside one
/// connection's own view, and what `find_<db>_snapshot` closes outright by wrapping the whole call
/// in the dialect's read transaction.
///
/// # An EMPTY page runs nothing
///
/// [`gathers`] renders none for one, so a query that matched nothing costs exactly the one
/// statement that found that out.
///
/// # Errors
///
/// A driver error, a lowering refusal carried verbatim, or a decode the plan's own crossing refused.
pub async
/// Take every aggregated collection off one page row, at the aliases [`aggregates`] wrote them
/// under.
///
/// # Errors
///
/// Whatever the plan's own decode refuses with.
pub
/// The ordinal's own suite — the sort-key-not-index pin, live. Beside the module's tests for the
/// reason the schema lane's shadow suite is beside its own: one property, held apart.