1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
//! Tier B: the [`AsyncGather`] seam a batch of keys crosses to entities through, the [`RetrieveError`]
//! the two halves of a retrieve refuse in, and the fill loop that composes them.
use Future;
use ;
/// The store side of a retrieve: a batch of keys, the caller's filter, and the entities that
/// survive both.
///
/// # It is a SEAM rather than an emitted statement, and that is what keeps the tier ungated
///
/// The composed query is `WHERE <row-value keys predicate> AND <the user's comparison>` — the
/// second leg of *search → keys → hydrate*, which is a shape the SQL family already renders. What
/// it cannot be is EMITTED at a derived index's declaration: a derived index names no database (it
/// stores no rows), the upstream's row types belong to another expansion that this one cannot see,
/// and an expansion that named `sqlx` would make a declaration in a build with no SQL family fail
/// to compile. So the composition is written ONCE behind this trait — which is the shape the
/// retrieve face was ruled with, the orchestration living in one framework-internal core and the
/// emitted methods delegating to it — and the framework ships the SQL implementation of it.
///
/// A third party implements this for its own store: a document database, a key-value cache, an
/// in-memory corpus in a test. Nothing about the retrieve pipeline changes when one arrives.
///
/// # The answer is POSITIONAL, which is what restores the engine's order
///
/// One slot per key, in the order the keys were given. Engine order is the list's order and there
/// is no `ORDER BY` anywhere on this path — a ranked list re-sorted by a column would be a ranked
/// list with the ranking thrown away.
///
/// [`None`] is *this key produced no entity*, and it covers TWO cases on purpose:
///
/// - the user's filter pruned the row;
/// - the store has no row at that key at all — the engine knows an address the system of record has
/// since lost.
///
/// They are one case here because they are one case to the caller: the page is short by one and the
/// fill loop pulls another key. Distinguishing them would be a channel for a condition nobody can
/// act on differently.
///
/// # No `Send` on the returned future
///
/// The store faces demand it because a graph's drivers are multi-threaded. A gather is called from
/// wherever the caller's retrieve runs, so requiring it here would refuse a perfectly good
/// single-threaded store for a property this path never spends.
/// Why a retrieve did not answer: the ENGINE refused, or the STORE did.
///
/// Two parameters and no flattening, because the two halves are two backends a caller chose
/// independently and each already has an error vocabulary of its own. A single boxed error here
/// would take a tantivy `QueryParserError` — which tells a client its query text was malformed —
/// and a `SqlQueryError` — which tells an operator a bind ceiling was hit — and make them one thing
/// nobody can match on.
/// The FILL LOOP: ranked keys in, ranked entities out, one composed query per batch.
///
/// # What it does, in the order it does it
///
/// The ranked keys arrive already floored, deduped and stamped — tier A's whole answer, unbounded,
/// because it is this loop's own filter that decides how many of them become results. Then:
///
/// 1. take the next batch of keys — as many as the page still needs;
/// 2. ONE call to [`AsyncGather::gather`], which is ONE statement: the keys predicate AND the filter;
/// 3. keep the slots that came back filled, each wearing the score and rank its KEY carried;
/// 4. if the page is still short, go back to (1).
///
/// The loop ends at `limit`, or when the keys run out. The keys run out for two reasons and the
/// loop cannot tell them apart, which is correct: either the engine's stream was exhausted, or the
/// floor cut it. Both mean *there is no more above-quality material*, and both legitimately leave a
/// page UNDER-FULL. 宁短不凑 — a short final page is the ruling, not a case to pad around.
///
/// # The batch is what the page still needs, and that is the whole sizing rule
///
/// No constant, no knob. An unfiltered retrieve fills in one round trip; a filter that prunes half
/// a batch asks for the shortfall and no more. The alternative — a fixed batch — is a number that
/// is too big for the first case and too small for the second, chosen once for every corpus.
///
/// # Rank and score come from the KEY, never from the row
///
/// [`Ranked::map_node`] is the one move, so the numbers on an entity are the engine's answer about
/// its address. A row carries no ranking and inventing one from its position in the page would
/// re-densify exactly the holes tier A was careful to leave.
pub async
/// The FILL LOOP AT THE SET GRAIN: [`filled`] over a [`Folded`] answer instead of a [`Ranked`] one.
///
/// Every word of [`filled`]'s note holds — the batch sizing, the positional contract, the short
/// final page, and the rule that the numbers come from the KEY and never from the row. The two
/// share [`hydrated`] rather than repeating the loop, because a loop written twice is a loop that
/// can drift at one of them, and the only difference between these two roads is WHAT rides beside
/// the key: two numbers on the single-vector road, and a folded score plus `m` per-stream sightings
/// here.
///
/// # What is carried is the whole provenance, not a re-derivation
///
/// [`Folded::map_node`] is the one move, exactly as [`Ranked::map_node`] is one road over: the
/// folded score and every [`StreamBest`](super::StreamBest) slot are the fold's answer about the
/// ADDRESS, and a row found under that address has nothing to say about either. A hit whose row the
/// system of record no longer holds drops out here, and the ordinals its slots carried stay holes in
/// the streams they came from — which is the same shape tier B already leaves in a `rank`.
pub async
/// THE FILL LOOP ITSELF — the batching, the positional read and the page cut, generic over what
/// rides beside a key.
///
/// Private, and the seam is deliberately a PAIR OF FUNCTIONS rather than a trait: the two callers
/// are this module's own, the split and the rejoin are two lines each, and a trait would put a
/// public vocabulary in front of an abstraction that exists to keep one `while` loop from being
/// written twice.
///
/// `C` is the hit WITH ITS NODE TAKEN OUT — a `Ranked<(), S>` on one road and a `Folded<(), S>` on
/// the other — and nothing in this body reads it. That is the property that makes the sharing safe:
/// the loop cannot pick a number up from the wrong hit, because it cannot see a number at all, and
/// the rejoin is each answer's own `map_node` rather than a constructor called with parts.
async