1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
//! What each term of a completed order SORTS BY — [`MongoSubjects`] — and the stages that put a
//! term's value ON the document before anything reads it.
//!
//! # The SQL family renders an expression; this one MATERIALIZES a field
//!
//! An ordering term over a to-one target's column lowers to a correlated scalar subquery in SQL, and
//! that expression is written three times: in the `SELECT` list, in the `ORDER BY`, and in every arm
//! of the resume predicate. MongoDB has no expression that reads another collection from inside a
//! `$sort` or a `$match`, so the value has to arrive on the document FIRST — and once it has, every
//! reader is naming a field.
//!
//! That is why this seam hands back a JOIN PLAN where
//! [`OrderSubjects`](crate::comparison::sql::OrderSubjects) hands back a rendering: the stages are
//! values, so the party that needs them takes them, and the three readers each take the one thing
//! they need, which is the KEY — see [`sort_key`], the single authority none of them can disagree
//! with.
//!
//! # The stages, for `cover.title` ascending with the `_id` tail
//!
//! ```text
//! { $lookup: { from: "keyframe", localField: "cover_id", foreignField: "_id",
//! as: "__ig_keyframe_1__" } }
//! { $addFields: { "__ig_ord0__": { $first: "$__ig_keyframe_1__.title" } } }
//! { $project: { "__ig_keyframe_1__": 0 } }
//! ```
//!
//! - **The `$lookup` body is the RELATION family's**, name for name — `Lookup` is reached from here
//! rather than restated — because a hop IS a join, and a second renderer would be a second place
//! for the pair of fields it meets on to be wrong. A hop passes no sub-pipeline: it asks the
//! related documents nothing, it fetches one value off them.
//! - **The join field is the TARGET's own**, taken from its
//! [`ALIASES`](super::MongoQuery::ALIASES) at the hop's depth, exactly as a relation at that depth
//! takes it. There is no second mint.
//! - **`$first`, because a forward reference is to-ONE.** `$lookup` always answers with an array, so
//! the value has to be taken out of one; the reference matches a unique key, so the array holds at
//! most one document and the first IS the value. A to-MANY position has no term that reaches here
//! — an order over one would have to say WHICH of the rows it meant.
//! - **`$project` drops the join field again**, which is the relation family's own third stage doing
//! the same two jobs: the joined documents are whole documents from another collection and nothing
//! downstream reads them, and consuming the field is what lets it be named by DEPTH alone.
//!
//! # One LEVEL is one self-contained run, and that is what makes the naming work
//!
//! The three stages above are written per ordering term, and each run drops its own fields before
//! the next opens. That is the relation family's arrangement verbatim, and it buys the same thing:
//! `cover.title` and `hero.title` are two terms reaching one collection at one depth, so they mint
//! ONE name — and two `$lookup` stages writing one document key would leave the second overwriting
//! the first with no error anywhere. Consumed per level, the repeat is harmless.
//!
//! What SURVIVES a run is the sort value alone, under [`sort_alias`], which is not a name any join
//! can take.
//!
//! # Depth is a CHAIN, and each hop is one name
//!
//! `cover.strip.label` is two hops, and the second one's `localField` reaches THROUGH the first
//! one's field:
//!
//! ```text
//! { $lookup: { from: "keyframe", localField: "cover_id", foreignField: "_id",
//! as: "__ig_keyframe_1__" } }
//! { $lookup: { from: "strip", localField: "__ig_keyframe_1__.strip_id", foreignField: "_id",
//! as: "__ig_strip_2__" } }
//! { $addFields: { "__ig_ord0__": { $first: "$__ig_strip_2__.label" } } }
//! { $project: { "__ig_keyframe_1__": 0, "__ig_strip_2__": 0 } }
//! ```
//!
//! A dotted `localField` resolving to an ARRAY is what makes the chain work without a sub-pipeline:
//! `$lookup` matches a document whose `foreignField` equals ANY element of it, and the array here
//! holds at most one element because the hop before it was to-one as well. The hops inside ONE run
//! are at distinct depths by construction, so their names differ even where every hop crosses into
//! the same collection.
//!
//! That is also what keeps a hop off `$lookup`'s COMBINED form — `localField`/`foreignField` beside
//! a `let` and a `pipeline` — which is the one MongoDB only learned in 5.0 and which a RELATION does
//! use, having a question to ask the joined documents. A hop asks none, so the chain uses nothing
//! the concise form has not had all along and a nested order puts no floor of its own under a
//! deployment. The 5.0 floor stays where [`lane`](super::lane) records it.
//!
//! **This is where the MongoDB family spends [`DEPTH_CEILING`](crate::query::DEPTH_CEILING).** The
//! SQL family's nested ordering spends none of it — a correlated subselect is its own SCOPE, so an
//! inner query may call its row source by the real table name and nothing has to be renamed — while
//! every stage of a pipeline writes into ONE document. Past the ceiling the hop refuses with
//! [`MongoLoweringError::RelationDepth`](crate::comparison::mongo::MongoLoweringError::RelationDepth),
//! which is the relation family's own refusal naming the same collection: a name that runs out is
//! one fact whichever kind asked for it.
//!
//! # A nested term is ALWAYS nullable, and both backends reach that by their own road
//!
//! A document whose reference is absent, and one whose reference points at nothing, both give the
//! `$lookup` an empty array — so `$first` has no element to take and the sort field is not written.
//! MongoDB orders a missing field exactly where it orders a `null`, which is where this framework
//! pins a missing value, so such a document sorts below every value on an ascending term. SQL gets
//! there differently and lands in the same place: a scalar subquery over no rows IS `NULL`.
//!
//! # What this costs, stated rather than apologised for
//!
//! **The `$sort` is a MEMORY SORT.** `__ig_ord0__` is computed by a stage, so no index covers it
//! and the server sorts the documents it has. That is the SAME COST CLASS as the SQL family's nested
//! ordering, which was landed on exactly these terms: `ORDER BY (SELECT …)` is a correlated
//! subquery evaluated per row followed by a sort over a value no index holds either. A nested order
//! is a sort by a column of another table, and neither engine has an index for that; what both have
//! is an honest plan for it.
//!
//! **These joins run BEFORE the `$match`, where a relation's run after it.** An ordering hop has to:
//! the resume predicate compares the value it materializes, so there is nothing to compare until
//! `$addFields` has written it. A relation's join is a conjunct and conjuncts commute, so it lands
//! after the `$match` and joins over fewer documents. MongoDB's own pipeline optimizer is documented
//! to split a `$match` and move the half that does not reference the joined fields in front of a
//! `$lookup`; this crate does not depend on that, and this repository provisions no server, so it is
//! recorded as what the documentation says and not as something measured here.
//!
//! **The join arrays never reach the wire.** `$project` drops them, so what comes back per document
//! is the sort value and not the joined documents it was taken from — which for a to-one reference
//! is one field instead of a whole row per hop.
use ;
use stage;
use crate::;
/// The `$addFields` stage's key — the one stage name this family writes and the relation family does
/// not, because materializing a value is what an ordering does and a filter has no use for.
const ADD_FIELDS: &str = "$addFields";
/// The aggregation expression that takes an array's first element — how a to-one join's array
/// becomes a value.
///
/// An ARRAY operator rather than the accumulator of the same name, and MongoDB learned it in 4.4 —
/// below the 5.0 floor [`lane`](super::lane) records, so it costs this lane no server version of its
/// own. `{$arrayElemAt: [<array>, 0]}` is the older spelling and is not carried beside it, for the
/// reason `$alwaysFalse` has no twin either: a lane that renders one expression on one server
/// version and another on the next is a lane whose pipeline a reader cannot check.
const FIRST: &str = "$first";
/// The `$project` value that DROPS a field, as opposed to the `1` that keeps one.
const DROP: i32 = 0;
/// The WORD a materialized sort value's field is built from — `__ig_ord0__` for the first term of
/// the completed order, `__ig_ord1__` for the second.
///
/// The SQL family spells the same name for the same two reasons: several of these ride ONE query,
/// so they have to be told apart, and the position in the order is what tells them apart — and one
/// spelling across the pair is the same one-authority law [`sort_key`] holds inside this family.
///
/// # It is the one minted name that SURVIVES its run
///
/// Every join field a relation or a hop opens is consumed by the `$project` beside it, which is what
/// lets those be named by DEPTH. This one cannot be: the `$sort` reads it, and the `$sort` is the
/// last thing that runs. So it is named by the TERM's position instead — and being a live key
/// beside the stored ones is what makes the law below load-bearing rather than tidy.
///
/// # THE LAW: a name under `__ig` is either the JOIN FRAME or a WORD, and a word carries no `_`
///
/// The framework mints two shapes in its reserved namespace:
///
/// ```text
/// frame __ig_<name>_<depth>__ the `_` before the digits is MANDATORY
/// word __ig_<word>__ the digits, if any, FUSE to the word
/// ```
///
/// A word is injective against every instance of the frame exactly when its body — what sits
/// between `__ig_` and the trailing `__` — contains no `_` at all: a frame instance's body always
/// splits at one, into a name and a run of digits, so a body with nothing to split on is a body no
/// frame can produce. `__ig_ord0__` is therefore reachable by no join, while `__ig_ord_0__` is
/// precisely the join a collection called `ord` opens at depth zero. A collection genuinely called
/// `ord0` is no counterexample: at depth two it mints `__ig_ord0_2__`, a different string again.
///
/// So the rule is one sentence and it governs every word this framework mints: **an `__ig` word's
/// tail must not look like `_<digits>__`.** `__ig_kind__`, `__ig_union__` and `__ig_source__` are
/// injective by having no digits at all; this one is injective by fusing them.
///
/// What that buys is why the name is here and not outside the namespace: a field spelled `__ord`
/// and a digit is one a declaration may legitimately own, and it stays the declaration's.
/// `ingraph-derive` refuses a declared name beginning `__ig` and refuses nothing else, so the
/// RESERVATION and the MINT now name the same set — where a mint outside it wrote a key the fence
/// never claimed, and `$addFields` would have overwritten a stored column of that name with no
/// error anywhere. That is the exposure
/// [`write_field_path`](crate::comparison::mongo::write_field_path) records for a dotted key,
/// closed here rather than recorded.
///
/// The word lives in the knobbed namespace: a deployment that moves the reserved prefix moves this
/// with it, since the prefix is the first thing the name is built from.
const SORT_ALIAS: &str = "__ig_ord";
/// What closes a minted word, and what makes its body a bounded thing to reason about.
const ALIAS_END: &str = "__";
/// The field term `level` of a completed order is materialized under.
///
/// Public because it is one end of a pair: the ENGINE writes this name into the pipeline and the
/// document reads its value back off the returned BSON at [`MongoOrdering::value`], so a page that
/// built the name itself would be free to build a different one.
///
/// [`MongoOrdering::value`]: super::MongoOrdering::value
/// ONE reference an ordering term crosses: the documents it points at, and the field holding their
/// id.
///
/// # The target arrives as the RELATION family's own value
///
/// [`MongoRelatedDocument`] carries the collection AND the field each nesting depth lands its
/// documents in, both read off one [`MongoQuery`](super::MongoQuery) binding — so a hop cannot name
/// one collection's address beside another's join fields. It is the same value a relation position
/// holds, and that is the point: a hop is a join, so it is described by the type a join is described
/// by.
///
/// # The far side of the match is DERIVED and never carried
///
/// A hop crosses a FORWARD reference, which points at the target's address, so the `foreignField` is
/// that address's one column as the target's own documents spell it —
/// [`field_key`](crate::comparison::mongo::field_key) is the authority, and it is `_id` for every
/// framework-addressed document. A target addressed by SEVERAL columns has no single field to match
/// on and is refused by the relation family's own width check, with the relation family's own
/// [`RelationWidth`](crate::comparison::mongo::MongoLoweringError::RelationWidth).
/// The path one ordering term takes to its value: the references it crosses, outermost first, and
/// the field it reads at the far end.
///
/// # It is NON-EMPTY by construction
///
/// [`new`](Self::new) takes one hop and [`via`](Self::via) adds another in front, so there is no way
/// to build a nesting that reaches nothing — which would be a term whose subject is the reference
/// COLUMN, a value no client ever asked to sort by. `via` prepends because that is the shape a
/// recursive order-by has: the outer variant holds the target's own order, so the hop nearest the
/// page is the last one known.
/// What each of a completed order's terms sorts by.
///
/// # It answers with a PLAN, where the SQL family's seam answers by rendering
///
/// [`OrderSubjects`](crate::comparison::sql::OrderSubjects) writes a subject's body into an open
/// buffer, because a SQL subject is several pushes carrying binds. Here the subject arrives as
/// stages, which are values, and every party that reads it afterwards is naming a field — so what
/// this hands over is the join, and [`sort_key`] is what turns it into the one name they share.
///
/// # The whole order arrives, not one term
///
/// `level` indexes `terms`, which is [`OrderSubjects`](crate::comparison::sql::OrderSubjects)'s own
/// shape and buys the same thing: a party that answers for some levels and defers the rest has both
/// halves in hand, and that mixed order is exactly what a nested ordering term produces.
/// Every term sorts by its own FIELD — the ordinary order, and the answer a document with no nested
/// ordering term states.
///
/// The MongoDB twin of [`TermColumns`](crate::comparison::sql::TermColumns), and it holds nothing
/// for the same reason: the terms arrive with the call.
;
/// A borrowed party answers exactly as the party does, so ONE value can subject every reading of a
/// page.
///
/// The `$sort`, the resume predicate and the stages that materialize the values have to agree about
/// which terms are nested and where each one's value lands, and they are three different callers —
/// so the engine builds one subjects value and hands `&S` to all three. Without this the keyset,
/// which holds its `S` by value, would take one away from the others.
/// The BSON key term `level` sorts by: the field the document already holds it in, or the alias the
/// join stages materialized it under.
///
/// **One authority for three readers.** The `$sort` entry, every equality leg of the resume
/// predicate and its comparison arm all go through this, so a page whose ordering named one field
/// and whose keyset named another is not a mistake anyone can make — which is the guarantee
/// [`MongoOrder`](super::MongoOrder) exists to hold, spelled at the level a nested term made
/// necessary.
pub Sized>
/// The stages a nested order needs before anything reads it, and the collections they add.
///
/// The two travel together because they are one fact: the party that ADDED a `$lookup` is the party
/// that names the collection it reads, which is what keeps
/// [`MongoPipeline::spans_collections`](super::MongoPipeline::spans_collections) — and therefore the
/// execution lane's read concern — from being a guess made after the pipeline was built. It is the
/// arrangement [`MongoConjunction::join`](crate::comparison::mongo::MongoConjunction::join) makes
/// for a relation, one layer over.
pub
/// The join stages for every nested term of `terms`, or why one of them has none.
///
/// # Errors
///
/// [`MongoLoweringError::RelationWidth`] for a hop whose target is addressed by several columns, and
/// [`MongoLoweringError::RelationDepth`] for a term crossing more references than that target's join
/// fields were minted for. Both are the relation family's own, raised in the relation family's own
/// order — the width first, because it is a fact about the reference rather than about the predicate
/// or the order that reached it.
pub Sized>
/// `{$first: "<path>"}` — the value a to-one join's array holds.