1pub const QUERY_SYNTAX: &str = r##"# query — OQX syntax reference
13
14The `query` tool takes ONE plain OQX string (+ optional `limit`/`cursor`, and
15`bare: true` to drop the automatic `_id`/`_path` envelope — needs a `select`).
16There is no {from, filter, order} envelope: every concern is a clause of the
17string. Two rules tell the two syntaxes apart: DOT navigation belongs to the
18host object model (`_doc.layer`, `_section._blocks`, `_dst._title`); WHITESPACE
19query directives belong to OQX — a nested query is `<receiver> collect|exists|
20count|first|single { <block> }` (a postfix directive + a `{ … }` block), never a
21method. Its distinctive power is the receiver-constrained nested query that
22correlates to the current row: `from docs where _nodes exists { where _kind ==
23"md:task" }` returns only the docs that themselves contain a matching node.
24
25 [select <items>] from <target> [where <pred>] [frontier <pred>] [follow <dest> [{ … }], …]
26 [flatten <expr> [{ … }] [as <name>]]… [group by <key>, …] [having <pred>]
27 [order by <expr> [asc|desc], …] [limit N] [offset N]
28 <target> count|exists|none|first|single { <block> } (scalar/one-row form: a bare target at the root scope)
29
30CLAUSE ORDER IS FIXED: select, from, where, frontier, follow, flatten, group by, having, order by, limit, offset —
31each at most once (`flatten` may repeat); an out-of-order clause is a parse error naming the order.
32Only `select` may drop its keyword, and only as the first clause
33(`_path, era from docs where …`). A top-level predicate needs `where`, because
34`where` follows `from` (`era > 1600 from docs` is the clause-order error); a
35block may LEAD with a predicate (see Sugar), but a predicate after a projection
36still needs the keyword (`{ _name where _kind == "md:task" }`, not
37`{ _name, _kind == "md:task" }`); `from docs count` is an error (write
38`docs count { … }`). `where` and later `select` items may use the same body's
39`select` aliases (`select _path, old: era < 1000 from docs where old`; an alias
40shadows a same-named field there). `and` / `or` / `not` are the connectives;
41the symbols `&&` / `||` / `!` are accepted synonyms (the words are canonical and
42what `print` writes).
43
44## Sugar (OQX 0.17)
45
46Every form here is shorthand for an explicit directive (the AST is the explicit form):
47 nodes { _kind == "md:task" } ≡ nodes collect { where _kind == "md:task" } — a receiver block with
48 no consumer is collect; a block whose LEADING expression is a predicate
49 (comparison, call, !/is/not, in, literal, (…), a consumer test — anything
50 but a bare name, a dotted path or a ^lift) is where-first. Names still
51 project: _nodes { _name } ≡ _nodes collect { _name }; filter a bare field with
52 _nodes { is checked }. Wherever a value stands (0.20), call arguments included:
53 size(_children { is checked }); x distinct { … } ≡ x collect distinct { … }. After a
54 follow destination too: follow _children { depth 2 } is that destination's block.
55 (a, b) a tuple (0.20): two or more expressions, concatenated one level flat — a value
56 (size((tags, aliases)), "x" in (tags, aliases)) and a receiver ((a, b) { … },
57 follow (before, after) { depth 2 }); (a) alone is grouping
58 refs(x)[0] ≡ refs(x) first { offset 0 } positional (integer literal or ${binding});
59 out of range ⇒ absent; refs(x)[0]._path navigates the row
60 ^docs[_path == ^company] ≡ ^docs first { where _path == ^company } first match or absent
61 ^docs[_path == ^company]! ≡ … single { … } required: exactly one, else filter_invalid
62 _title! required: _title, or an error naming the expression (and the row's $key — its _id)
63 — never a filter, never a coercion (0!, ""! are values); tightest: a!.b vs a.b!
64 is x / not x truthiness of x, and its negation (prefix; `not` ≡ `!`)
65 x is y / x is not y identity ($key: a store row's id, else structural) — compares rows, which == does not;
66 x is null = absent; for scalars is ≡ ==; comparison precedence, no chaining
67 a and b / a or b the connectives (≡ && / ||: same precedence, short-circuit, value: title or _title coalesces)
68 boss: ^docs[_path == ^manager], bossName: boss._title a select item may use the items to its LEFT
69 Reserved words (never a bare field name): from where select is not and or, true false null.
70 Contextual words (limit order offset follow collect count first distinct desc by in frontier depth group having
71 flatten as) parse as fields in expression position (select o: order from docs).
72
73Results are LEAN hits — { _id, _path } + whatever `select` projects (or `values`,
74`count`, `exists`, `none`). The envelope keys ARE the attributes: copy a hit's
75`_id` straight into `where _id == "…"` (no translation); `select _id, _path` is
76simply the envelope (no duplicate columns); an alias `_id:`/`_path:` on any OTHER
77expression is refused — `select _path: _doc._path from blocks` fails ("projection
78`_path` collides with the hit envelope — alias it (`p: _doc._path`)"); a row
79surfacing as a value (an empty `collect { }`) is the same `{ _id, _path }`. Every
80other tool spells a row the same way (`docs_read` → `{ _id, _path, rev, properties,
81_raw }`, `blocks_read` → `{ _id, _type, _text, _raw, _hash, … }`, `docs_list` → `{ _id,
82_path, _updated_at, blocks }`, a revision → `{ _id, _path, _hash }`): hydrate by `_id`
83via blocks_read/docs_read.
84
85## Paths
86
87Every path the surface returns is `/`-rooted — the form a reference is
88written in (`[x](/projects/oqx.md)`, `before: [/timeline/kickoff.md]`): a
89hit's `_path`, an edge's `_dst_path`, every tool's paths. Every path it
90accepts tolerates both forms (`/a.md` or `a.md`): tool arguments, `refs(x)`,
91`within(...)`, and a string LITERAL compared with `_path`/`_dst_path` (on the row,
92or through a row handle: `_doc._path`, `_dst._path`, `$it._path`) by `==`/`!=` or
93passed to their `.startsWith(...)` (it is rooted first, so `_path == "a.md"` and
94`_path == "/a.md"` both match). A property holding a reference compares directly —
95and is never rooted: `where ^_path in list(after)`, `where customer == ^_path`,
96`where path == "/x.md"` (YOUR authored `path:` key — the bare name — compared as
97written) — no `"/" + ...` glue.
98
99## Targets & field namespaces
100
101THE NAME RULE — one rule on every record kind: a BARE name is YOURS (an authored
102property: frontmatter + inline on a doc, an attrs key on a block or node), an
103UNDERSCORE-prefixed name is the ENGINE's (however derived: `_id`, `_path`,
104`_title`, `_updated_at`, `_doc`), and `$` belongs to OQX — $it, $key, $depth,
105$stop, $leaf, $frontier, $ordinal are values the LANGUAGE injects into a scope;
106omgbase adds NO $ names (a `$id` in a query is an unknown name, absent). Nothing
107bare is reserved: `title`, `tags`, `path`, `id`, `format`, `doc` are your keys
108when you wrote them. Collections stay bare (`from docs`, `^docs`, `docs count { … }`,
109`entries(^$it)`) — `from` names a collection, not a row property — and so do the
110row functions in call position (`text("x")`, `under(…)`, `refs(x)`): a call is
111never an entry read. There is NO self alias (the 4.x `doc`/`block`/`section`):
112`$it` is the self, and `_doc`/`_block`/`_dst` reach ANOTHER row. So `_path` is the
113engine's /-rooted location and `path` your `path:` key; `_tags` is your tags ∪ the
114body #hashtags and `tags` the authored list alone; `_title` the computed title (the
115first H1) and `title` your `title:`; `_data.<k>` ≡ `<k>` (the merged authored view —
116kept for `entries(_data)` and to reach an authored key that itself starts with
117`_`), `_frontmatter.<k>` / `_inline.<k>` one source each. Same value, same name
118everywhere: a hit's `_id` is docs_read's `_id` and the `_id` argument of blocks_read.
119
120 Engine names per kind (attributes | handles | relations):
121 docs _id _path _raw _hash _updated_at _created_at _title _tags _format | _data _frontmatter _inline | _blocks _nodes _out_links _in_links _revisions
122 blocks _id _path _doc_id _raw _hash _updated_at _created_at _type _text _attrs _depth _ordinal | _doc | _children _nodes _out_links _section
123 nodes _id _path _doc_id _block_id _kind _name _value _attrs | _doc _block | _blocks _subsections _children
124 links _id _path _doc_id _block_id _via_id _dst_id _dst_path _uri _field _dst_kind _predicate _provenance _anchor | _doc _dst _block | —
125 revisions _id _path _rev _seq _doc_id _commit _ts _origin _actor _reason _hash _is_current _archived | _doc | —
126 root _id (^_id, 0^_id) | — | docs blocks nodes links revisions
127 Every bare name is yours; `_attrs.<k>` on a block or node is the same read as
128 the bare attrs key `<k>`.
129 UNKNOWN NAMES: a bare name that no object of the kind in this repository
130 carries is a property read that yields absent — on docs/blocks/nodes the result
131 gains `hints` (one line per name, the nearest match drawn from your keys and the
132 engine names: "no document in this repository carries a property `title`; did
133 you mean `_title`?") and the rows still come back; in a plan/apply/execute
134 selector it is refused. On links and revisions (closed: no authored data reaches
135 them) a bare name is a filter_invalid error. A `_x` that is no engine name of
136 the kind is filter_invalid naming the kind's engine names. A property some
137 objects carry is known (partial presence is normal).
138
139docs:
140 - bare identifier = a document PROPERTY (frontmatter + inline `key:: value`,
141 unioned; nested via dots: meta.owner). YAML/JSON docs
142 expose the parsed object's keys the same way. Nothing is
143 reserved: `title`, `tags`, `path`, `id` are your keys.
144 - source handles = _data.<k> (the merged authored view — the same read as the
145 bare key) / _frontmatter.<k> / _inline.<k>; the whole bag
146 as a collection via entries(_data) / entries(_frontmatter)
147 / entries(_inline) (see entries()).
148 - attributes = _id, _path (/-rooted, see Paths), _updated_at (ISO-8601,
149 sorts chronologically), _created_at, _raw (the whole file,
150 frontmatter included), _hash (its sha256, 64 hex), _title
151 (the computed title: the first H1; `title` is the authored
152 key), _tags (authored tags ∪ body #hashtags, deduped,
153 authored first; `tags` is the authored list), _format (the
154 doc's format — "markdown"/"yaml"/"json"; present on EVERY
155 doc, so has(_format) is always true and `_format == "yaml"`
156 is the filter).
157 - relations = _nodes, _blocks, _out_links / _in_links (a doc's links as
158 rows; the documents at their other end are
159 _out_links collect distinct { from _dst } and
160 _in_links collect distinct { from _doc } — there is no
161 out / in relation), _revisions (the document's stored
162 revisions, oldest first — see revisions below).
163
164blocks:
165 - bare fields = every attrs key FLATTENED onto the row (bare `checked`;
166 `_attrs.<k>` is the same read; absent ⇒ false) — a block's
167 authored side.
168 - attributes = _id, _path (the owning doc's path), _doc_id (the owning
169 doc), _type, _text (normalized), _ordinal (0-based among
170 siblings), _depth (0 at the top level), _raw (the block's
171 raw source — a container's raw holds its children), _hash
172 (its sha256 — the value update/split expect as
173 expect.content_hash), _updated_at, _created_at, _attrs.
174 - reach-through = _doc.<key> / _doc._path / _doc._id / _doc._format = the
175 CONTAINING doc (_doc._id and _doc._path cost nothing — they
176 read the row's own _doc_id / joined path, no lookup).
177 - relations = _children, _nodes, _out_links, _section (no self alias:
178 the 4.x `block.children` is `_children`).
179 NOTE: inside a `follow`, $depth/$ordinal are ALWAYS the walk's metadata; a
180 block's own nesting is `_depth` / `_ordinal` everywhere.
181
182nodes:
183 - bare fields = the attrs keys flattened onto the row (checked, level,
184 first_ordinal; `_attrs.<k>` is the same read).
185 - attributes = _id, _path, _doc_id, _block_id (null for a block-less
186 node), _kind (md:task / md:link / md:section / yaml:…),
187 _name, _value, _attrs.
188 - reach-through = _doc.<key> / _doc._id / _doc._path, _block._type /
189 _block._text / _block._id / _block._path.
190 - relations = _blocks, _children, _subsections (a section node's; the
191 4.x `section.blocks` is `_blocks`).
192
193links: (the authored link graph as rows — one per open link: the relation a link
194 establishes; the md:link NODE is its occurrence in the text and _via_id
195 points at it — a frontmatter link has a row and no node)
196 - attributes = _id, _path (the SOURCE doc's path), _doc_id (the SOURCE
197 doc), _block_id (the source block; null for a frontmatter
198 edge), _via_id, _dst_id (the destination node id: a doc id,
199 an external node id or phantom:<path>), _dst_path (target
200 doc path, /-rooted; NULL when dangling/external), _uri
201 (external URL; NULL otherwise), _predicate
202 ("references"/"embeds"/freeform), _provenance
203 ("link"/"frontmatter"/"inline_field"/…), _dst_kind
204 ("document"/"external"/"collection"), _anchor, _field (the
205 frontmatter key / inline field it was authored in).
206 - reach-through = _doc.<key> (the source document), _dst.<key> (the
207 destination document; absent when dangling/external),
208 _block.<key> (the source block). Closed: links carry no
209 authored data, so a bare name is filter_invalid. There is
210 no from_commit: a query sees open edges only (history is
211 blocks_history/changes_since).
212 - text()/semantic() are N/A on links.
213
214revisions: (history as rows — one per stored revision of every document of the
215 repository, deleted documents included: what docs_history lists, queryable)
216 - attributes = _id (the revision id, r_…), _rev (= _id), _seq (1, 2, … per
217 document), _doc_id, _commit, _ts (the commit's time, sorts
218 chronologically), _origin ("observed"/"api"/"import"/
219 "projection"), _actor, _reason, _hash (the rendered file's
220 sha256 — docs_history's _hash), _is_current (true for the
221 document's current revision), _archived (true when the
222 bodies moved to an archive segment: the row stays,
223 docs_read { at } reads through the segment), _path (the
224 path the revision was committed under — a move re-paths
225 only the current revision, so earlier ones keep the old
226 path).
227 - reach-through = _doc.<key> reaches the document's CURRENT row; absent for
228 a deleted document, so where _doc == null is the revisions
229 of deleted documents. Closed: a bare name is filter_invalid.
230 - on docs = _revisions is a relation (oldest first): a document with
231 more than one version is docs { _revisions count { } > 1 }.
232 - text()/semantic() are N/A on revisions. The body as of a revision is
233 docs_read { _path, at: "<rev>" } (docs_read_at is the same call).
234
235## Scoping (ADR-015; the root row since surface 2.0): bare names are LOCAL
236
237 name the CURRENT row only — never falls through to an enclosing row
238 or the repository; an absent field is simply absent.
239 ^name one scope OUT (the enclosing row's fields/intrinsics/lifts);
240 ^^name two scopes out. `^_path` = the enclosing row's path.
241 ^docs ^nodes ^blocks ^links ^revisions
242 THE ROOT ROW (surface 2.0, oqx 0.18): the repository is the root
243 scope's row, one scope out from a top-level row, so ^docs there is
244 the whole documents scan (uncorrelated until you add a ^ predicate);
245 one caret per enclosing block (^^docs from depth two), or the
246 absolute 0^docs from ANY depth. ^_id / 0^_id = the repository id.
247 ^$it = the root object: ^$it.docs ≡ ^docs, entries(^$it) names the
248 five collections. At the root scope itself a bare target is the scan
249 (docs count { … }, from docs, entries(docs)).
250 ERRORS: a bare `docs` INSIDE a block reads a property of the current
251 row, which has none — refused ("did you mean ^docs"); so is a bare
252 `blocks`/`nodes`/`revisions` there (a doc's own relations are
253 `_blocks`/`_nodes`/`_revisions`); `^docs` AT THE TOP LEVEL reaches
254 past the root — refused (write the bare `docs`); so is `$repo`
255 (surface < 2.0; the message names the replacement).
256 $it the current item itself (a row, or each ELEMENT when the
257 receiver is a list property); ^$it = the enclosing row, ^$it._id
258 its id. $value is NOT an intrinsic (since oqx 0.15): it reads a
259 property literally named $value, absent when there is none.
260 $key the row's identity (0.20): a store row's id (doc/block/node/edge), an entries(x)
261 entry's key, a follow occurrence's walk identity; project $key: <expr> to rekey the
262 rows a block yields (a destination block's $key: is not a column).
263
264## Scalar semantics (@omgbase/oqx)
265
266 comparisons == != < <= > >= strict & typed (5 == "5" is false); either
267 side may be a field (field-vs-field OK).
268 absence null and a missing field are the same "absent" value; two absents
269 are EQUAL, so `absent != "x"` is TRUE and `absent == null` is TRUE.
270 Relational (< <= > >=) on an absent operand → FALSE. Bare `f` in
271 boolean position uses JS truthiness (absent/false/0/"" ⇒ false;
272 NOTE: "false", [] and {} are truthy). has(f) = explicit presence.
273 boolean and or not grouping ( … ) — && || ! are accepted synonyms
274 arithmetic + - * / % (+ concatenates if either side is a string)
275 membership x in y array → typed membership; string → substring;
276 object → key exists; range → coverage. Any list
277 property works directly: "pricing" in tags. list(f)
278 coerces scalar-or-list (absent → []) — use it when a
279 key is sometimes scalar, sometimes a list.
280 ranges lo..hi (inclusive) lo...hi (exclusive hi) ..hi lo..
281 era in 800..1680; works over ISO dates. range(prop) reads a
282 range-shaped STRING property as an interval: "2026-01-15" in range(window).
283 strings x.contains("s") x.startsWith("p") x.endsWith("s") x.matches("^re$")
284 (real regex) x.lower() x.upper() size(x). ALL CASE-SENSITIVE —
285 fold with .lower(): _title.lower().contains("aurora").
286 Infix regex: x ~ "re" is x.matches("re"); x ~* "re" is the
287 case-insensitive match; x !~ "re" and x !~* "re" negate them.
288 Unanchored unless the pattern anchors; an absent x is false
289 under ~ and TRUE under !~ (not SQL's NULL).
290 functions has(f) size(x) list(x) entries(x) range(s)
291 dates a date is an ISO string, a duration a NUMBER OF SECONDS; arithmetic
292 is METHODS, never + or - (+ concatenates strings).
293 date("today"|"now"|"yesterday"|"tomorrow"|"sow"|"eow"|"som"|"eom"|
294 "soy"|"eoy") — the host clock, UTC; date("2026-10-15") canonical ISO
295 (absent if unparseable). dur("1h30m") dur("2 days, 3 hours")
296 dur("P1DT2H") dur(90) → seconds (year = 365 d, month = 30 d).
297 d.plus(dur("30 days")) d.minus(86400) d.plus("1 month") (calendar:
298 Jan 31 → Feb 28) a.until(b) → signed seconds d.year() d.month()
299 d.day() d.weekday() (1 = Mon) d.hour() d.minute() d.second()
300 d.date() d.format("EEE dd MMM yyyy") n.dur_text() → "1h 30m"
301 n.dur_iso() → "PT1H30M". Total: a non-date is absent, so
302 `where date(due) < date("today").minus(dur("30 days"))` just
303 filters. A datetime is never < its own date: compare
304 _updated_at.date() == date("today"), or < date("tomorrow").
305 literals "str" 'str' 123 1.5 true false null
306 determinism no random; the only clock is date("today"/"now"/…), read once per
307 query. Lists/structs are not literals; ternary is not supported.
308 Errors surface as filter_invalid.
309 scalars vs lists a list-authored key never == a scalar (tags == "x" is false
310 when tags is a list) — use `"x" in tags` / `in list(tags)`.
311
312## Domain predicates (omgbase)
313
314 text("terms") FTS5 PRUNING predicate (docs match via their blocks, nodes
315 via the node index); composes anywhere, incl. nested scopes.
316 semantic("phrase") embedding cosine SCORE (docs/blocks): threshold it
317 (semantic("x") > 0.6), project it, or `order by semantic("x")
318 desc` for top-K. Needs a provider that has embedded the repository; else provider_unavailable.
319 blocks target only:
320 under(id_or_heading) block in the containment subtree / section range
321 under_heading("Launch") an ancestor section heading contains the text (ci)
322 under_kind(type[, name]) an ancestor block has that type (name: its text
323 contains / its key equals name)
324 within("path" | "d_..") containing doc = id, exact path, or path glob (*);
325 either path form
326 yaml_path("a.b") YAML docs: the block at that key path
327 json_pointer("#/a/b") JSON docs: the block at that pointer
328 has_edge("pred"[, target]) an open authored edge with that predicate leaves here
329 has_anchor() block carries an authored ^ref
330 parent_type() == "x" parent block's type (must be compared)
331 child_count() > 0 number of direct children (must be compared)
332 refs(x) the live documents a property's document references name:
333 x is a string, a list or absent; each "/a/b.md", "a/b.md"
334 or "d_…" element resolves to that doc's row; dangling and
335 non-string elements are dropped. Yields docs rows, so it is
336 a source, a receiver or a follow destination:
337 follow refs(before), refs(after) walks a timeline both ways.
338 The reverse direction needs no function: a document's
339 _path is already the reference form, so
340 ^docs { where ^_path in list(after) } is
341 "the documents whose `after` names me".
342 A HIT IS A STORE ROW: a top-level row that is not a doc/block/node/edge fails
343 (filter_invalid). `follow before` over a list of paths
344 reaches the STRINGS — write `follow refs(before)`.
345
346## Nested queries & consumers
347
348 <receiver> collect|exists|none|count|first|single { <block> }
349 receiver = a relation (_nodes, _blocks, _doc._out_links, _section._blocks, …), a
350 reach-through row (_dst, _doc: one row — _dst { _title }), a
351 list property (tags), entries(x), or a root collection (^docs; 0^docs at depth ≥ 2).
352 block = [select …] [from E] [where …] [frontier …] [follow …] [flatten …] [group by …] [having …] [order by …] [limit N] [offset N]
353 — the same fixed order as the top level (`from` optional; the
354 receiver supplies the rows). A leading name/alias list is the
355 projection (`select` dropped); a predicate always needs `where`.
356 A `from E` inside a block RE-PROJECTS the rows through a relation
357 (a typed flatMap): `docs collect { from _nodes where _kind == "md:task" }`
358 starts at docs, projects each through `_nodes`, and queries the
359 resulting nodes; a scalar relation (`from _doc`) contributes one
360 row; a missing or non-navigable relation is a loud error, never a
361 global scan.
362 exists ≥1 row _nodes exists { where _kind == "md:task" and not checked }
363 none 0 rows _nodes none { where _kind == "md:task" and not checked } ("every task done")
364 count a number _nodes count { where _kind == "md:task" } >= 2
365 collect rows (default) — in select for nested results; in where with a lift
366 first / single zero-or-one (single errors on >1) — lookups in select
367 distinct on any consumer: dedup by projected value — _nodes count distinct { select _kind }
368 limit/offset bound the block's rows AFTER where/order/distinct, BEFORE the
369 consumer reduces: _nodes exists { offset 1 } = at least two;
370 first { … offset 1 } = the second.
371 lift `^name:` in a where-collect binds values outward:
372 select _path, open from docs where _nodes collect { ^open: _value where _kind == "md:task" and not checked }
373 joins ^<t> exists { where slug == ^ref } semi-join; not … exists anti-join;
374 ^nodes single { where _kind == "person" and _attrs.id == ^owner_id } lookup;
375 x in ^keys membership over a lifted set.
376
377## select — projection
378
379 select a, alias: expr, nested: _nodes collect { … }
380 bare key → your property (scalar-authored → scalar; list → array); _name →
381 the engine's field (_id, _path, _title, …); _data.<k> is the bare key's read;
382 a call/arithmetic needs an alias (n: size(tags)) unless in values mode;
383 an output key of _id/_path other than the attribute itself needs an alias.
384 _raw whole file bytes (docs) — docs_read is cheaper for one doc.
385 distinct select distinct _type (dedup hits by projected value)
386 values select <ONE item> values → bare values instead of records:
387 select distinct type values from docs → values: ["hub","lab-note",…]
388 select _path values from docs → values: ["/index.md", …]
389 tags: tags collect { $it values } → a plain array
390 h: _nodes first { _name values where _kind == "md:section" order by first_ordinal }
391 entries(x) a record as a collection ($key / $it per entry), key order:
392 select fm: entries(_data) collect { k: $key, v: $it }
393 where entries(_frontmatter) exists { where $key == "era" and $it > 1600 }
394 entries(_inline), entries(_attrs), a nested map, or a list (index keys)
395 $semantic_score is not a field — project semantic("…") under an alias instead.
396
397## flatten (OQX 0.25)
398
399 flatten <expr> [{ … }] [as <name>] after follow, before group by; may repeat
400 Splits one value into rows. Every hit is still the PARENT row — its _id/_path
401 repeat per element, as Dataview's TABLE … FLATTEN — with <name> bound beside
402 its fields at the same depth: an array → one row per element (an empty list
403 yields none), a missing field → one row with the name absent (never dropped),
404 an object or a scalar → one row with the value. A plain navigation names
405 itself (flatten tags rebinds tags to each element); anything else needs as.
406 The block is the ordinary receiver block over the elements (where, a
407 projection, order by, limit, distinct), applied first. Later clauses read the
408 name — a later flatten (flatten jobs as j flatten j.tags as t), group by,
409 having, select, order by; where runs before it. A flattened query pages by
410 position (cursor null); as _id / as _path is refused (the envelope reads them).
411 select _path, tags from docs where type == "process" flatten tags order by _path, tags
412 select _path, t: b._text from docs flatten _blocks { where _type == "task" and not checked } as b
413 select tags, n: count() from docs flatten tags group by tags order by count() desc
414
415## order by, limit/offset, pagination
416
417 order by era desc, _path asc absent values sort LAST (asc); ties → (_path, _id)
418 limit N / offset N define the result SET (integer literals);
419 the tool's limit/cursor then page WITHIN it
420 cursor opaque keyset on (_path, _id); a custom order by
421 disables it (you get the top page + truncated)
422
423## Graph traversal (follow) and links
424
425 The citation graph is the LINK ROWS walked through a destination block:
426 _dst is a link's destination document (absent for an external or dangling
427 link — it then contributes no successor), _doc its source.
428 from docs where <seed> follow _out_links collect { from _dst } docs the seed links TO
429 from docs where <seed> follow _in_links collect { from _doc } backlinks
430 follow _out_links collect { from _dst depth 3 } bound (1..8, default 8) — a clause of the block
431 follow _out_links collect { from _dst where layer == "canon" } keep only matching successors
432 follow _in_links collect { from _doc where ^_path in list(before) } ^ = the row being expanded
433 (^^ = the walk's enclosing scope)
434 follow _out_links collect { from _dst where _predicate == "depends_on" } one predicate only
435 follow _out_links collect { from _dst }, _in_links collect { from _doc }
436 several destinations: their union
437 (one step dedups by identity)
438 follow ^docs { where _out_links exists { where _dst_path == ^^_path } }
439 a destination block over the root's
440 documents, re-read per frontier row
441 (^ = that row) = the backlinks walk
442 follow _out_links collect { from _dst frontier type == "practitioner" } cut, keeping the frontier row
443 from docs where type == "process" frontier !verified follow _out_links collect { from _dst }
444 the BODY's frontier applies to the seeds:
445 an unverified seed is kept, not expanded
446 follow distinct _out_links collect { from _dst } / … { $key: group from _dst } identity control ($key)
447 follow refs(before), ^docs { where ^_path in list(after) } forward over `before`, backward
448 over every doc whose `after` names me
449 Per-occurrence metadata: $depth (seed = 1), $stop (interior|leaf|frontier|
450 depth|cycle), $leaf/$frontier, $ordinal (a deterministic 1..N rank over the
451 walk, ordered by depth then path — `order by $ordinal` for walk order,
452 `where $ordinal <= N` for a deterministic budget cut), $key — usable in
453 select/order by and the top-level post-walk where (a where conjunct that
454 reads one filters the walk's RESULT; the other conjuncts remain the seed
455 predicate), NOT metadata of the candidates a destination block's where reads,
456 nor valid inside a consumer directive. Cyclic graphs are safe: a revisited
457 node is admitted once as a `$stop == "cycle"` occurrence and never
458 re-expanded. The default keeps one occurrence per distinct walk path;
459 `follow distinct` dedups reached rows by identity.
460 Destination blocks (OQX 0.20): a brace after ANY destination opens that
461 destination's block — _children { depth 2 }, refs(x) { depth 2 }, _out_links
462 collect { from _dst depth 2 }, ^docs { where ^_path in list(after) } — with the
463 0.17 body rules (where-first, projections) and two clauses of its own after
464 `where`: frontier <pred> (occurrences reached through this destination that
465 match are frontiers; ^ there is the walk's enclosing scope) and depth <n>
466 (this destination expands a frontier row only while $depth < n). The body
467 that carries the follow may write frontier <pred> too, after its where and
468 before follow: it applies to the SEEDS (read like the body's where) — a
469 matching seed stays in the result with $stop == "frontier" and is not
470 expanded; each destination's frontier is its own. There is no options block;
471 the old `by` is $key. A bare follow ^docs (no brace, no consumer) is an
472 error naming both block spellings.
473 Type-preserving relations: a block's _children; a section node's _children and
474 _subsections; the citation graph is _out_links collect { from _dst } / _in_links collect { from _doc }.
475 from links where _dst_path == "/notes/x.md" inbound links to a doc, as rows
476 select d: _dst._title, k: _dst_kind from links the destination document's title
477 from docs where _out_links exists { where _dst.layer == "draft" } links into drafts
478 select b: _in_links collect distinct { from _doc } from docs backlinks as { _id, _path }
479 select _doc_id, _dst_path from links where _predicate == "depends_on"
480 from docs where not _in_links exists { } orphans (nothing links in)
481 (The `graph` tool is a convenience wrapper that compiles to a follow query.)
482
483## semantic grain
484
485from docs ranks whole DOCUMENTS (one hit per file); from blocks ranks PASSAGES.
486text()/where prune the candidate set; only order by reweights.
487
488## Examples
489
490 from docs where layer == "working"
491 from docs where _path.startsWith("/guides/") and _updated_at >= "2026-08-01"
492 select layer, tags from docs where "pricing" in list(tags)
493 from docs where _inline.owner == "alice"
494 from docs where _title.lower().contains("q3 plan")
495 from docs where _format == "yaml"
496 from docs where type == "practitioner" order by era desc limit 2
497 from docs where _nodes none { where _kind == "md:task" and not checked }
498 from docs where tags exists { where $it == "tria-prima" }
499 select fm: entries(_data) collect { k: $key, v: $it } from docs where _path == "/x.md"
500 select owner_id, owner: ^nodes single { where _kind == "person" and _attrs.id == ^owner_id } from docs
501 from blocks where _type == "task" and not checked and under_heading("Launch") and _doc.layer == "working"
502 from blocks where _type == "paragraph" and has_edge("references", "d_92aaaaa")
503 select _path, $depth from docs where _path == "/timeline/review.md" follow refs(before), refs(after)
504 select _path, next: ^docs collect { _path where ^_path in list(before) } from docs where type == "milestone"
505 select _path, n: size(^docs), repo: ^_id from docs limit 1
506 select path, title from docs where has(path)
507 docs { type == "substance" and not verified }
508 select subject, process: ^docs[slug == ^subject]!._path from docs where type == "lab-note"
509 select _path, back: _nodes first { select p: 0^docs collect { _path where ^^_path in list(after) } values } from docs where type == "milestone"
510 select _path, prior: refs(before) collect { _path, when } from docs where type == "milestone"
511 from blocks order by semantic("identity preservation across edits") desc limit 10
512 select _uri from links where _dst_kind == "external"
513 docs count { where layer == "canon" }
514 select _seq, _origin, _ts, _hash from revisions where _path == "/processes/calcination.md" order by _seq desc
515 from docs where _revisions count { where _origin == "api" } > 0
516 select p: _path, d: _doc._path from revisions where _doc == null
517 select layer, n: count(), titles: collect(_title), open: rows count { where _nodes exists { where _kind == "md:task" and not checked } } from docs group by layer having count() > 1 order by count() desc
518 select _path, tag: tags from docs where "process" in tags flatten tags order by _path, tags
519
520Common mistake: `$path`/`$id` — omgbase has no $ names; the engine's location and
521identity are `_path`/`_id`. Another: the 4.x bare `path`/`id`/`title`/`tags` — a
522bare `path` is now YOUR `path:` key (absent when you wrote none, with a hint
523pointing at `_path`); the engine's are `_path`/`_id`/`_title`/`_tags`; `select
524data.path` is gone — write `path`. Another: `where tags == "x"` on a list-valued
525key is false — write `"x" in tags`. Another: `select _path: _doc._path` — alias it
526(`p: _doc._path`).
527"##;
528
529pub const QUERY_SYNTAX_SECTIONS: &[(&str, &str)] = &[
536 ("sugar", "Sugar (OQX 0.17)"),
537 ("paths", "Paths"),
538 ("targets", "Targets & field namespaces"),
539 (
540 "scoping",
541 "Scoping (ADR-015; the root row since surface 2.0): bare names are LOCAL",
542 ),
543 ("scalars", "Scalar semantics (@omgbase/oqx)"),
544 ("predicates", "Domain predicates (omgbase)"),
545 ("nested", "Nested queries & consumers"),
546 ("select", "select — projection"),
547 ("flatten", "flatten (OQX 0.25)"),
548 ("order", "order by, limit/offset, pagination"),
549 ("follow", "Graph traversal (follow) and links"),
550 ("semantic", "semantic grain"),
551 ("examples", "Examples"),
552];
553
554pub const QUERY_SYNTAX_SECTION_NAMES: &[&str] = &[
556 "sugar",
557 "paths",
558 "targets",
559 "scoping",
560 "scalars",
561 "predicates",
562 "nested",
563 "select",
564 "flatten",
565 "order",
566 "follow",
567 "semantic",
568 "examples",
569 "all",
570];
571
572#[must_use]
577pub fn query_syntax_section(slug: &str) -> Option<String> {
578 if slug == "all" {
579 return Some(QUERY_SYNTAX.to_owned());
580 }
581 let (_, heading) = QUERY_SYNTAX_SECTIONS.iter().find(|(s, _)| *s == slug)?;
582 let lines: Vec<&str> = QUERY_SYNTAX.split('\n').collect();
583 let title = format!("## {heading}");
584 let start = lines.iter().position(|l| *l == title)?;
585 let mut end = start + 1;
586 while end < lines.len() && !lines[end].starts_with("## ") {
587 end += 1;
588 }
589 while end > start + 1 && lines[end - 1].trim().is_empty() {
590 end -= 1;
591 }
592 let mut text = lines[start..end].join("\n");
593 text.push('\n');
594 Some(text)
595}
596
597pub const QUERY_SYNTAX_CARD: &str = r#"# query — OQX card. Chapters: query_syntax { section: "<name>" }; the manual: { section: "all" }
602
603One OQX string; limit/cursor page the hits; bare: true drops the { _id, _path } envelope.
604 [select <items>] from <target> [where <pred>] [follow <dest> { … }] [order by <expr> [desc]] [limit N] [offset N]
605 <target> count|exists|none|first|single { <block> } a scalar or one row
606CLAUSE ORDER IS FIXED: select, from, where, frontier, follow, flatten, group by, having, order by, limit, offset.
607and / or / not. Comparisons strict and typed; null == a missing field. A list key: "x" in tags, never tags == "x".
608Strings .contains("s") .startsWith("p") .lower(); x ~* "re" (ci regex), x !~ "re". Dates date("today"),
609date(due) < date("today").minus(dur("30 days")), _updated_at >= "2026-08-01".
610
611NAMES: a BARE name is YOURS (frontmatter, inline key:: value, a block's attrs); an UNDERSCORE name is the ENGINE's;
612$ is OQX's ($it $key $depth $stop $ordinal). title tags path id are your keys; _title _tags _path _id the engine's.
613 docs _id _path _raw _hash _updated_at _created_at _title _tags _format | _data _frontmatter _inline | _blocks _nodes _out_links _in_links _revisions
614 blocks _id _path _doc_id _raw _hash _type _text _attrs _depth _ordinal | _doc | _children _nodes _out_links _section
615 nodes _id _path _doc_id _block_id _kind _name _value _attrs | _doc _block | _blocks _subsections _children
616 links _id _path _doc_id _block_id _via_id _dst_id _dst_path _uri _dst_kind _predicate _provenance _anchor _field | _doc _dst _block
617 revisions _id _path _rev _seq _doc_id _commit _ts _origin _actor _reason _hash _is_current _archived | _doc
618Handles _data.<k> ≡ <k>, _frontmatter.<k> / _inline.<k>; entries(_data) the bag as rows ($key, $it). Reach-through
619_doc.<k> the owning document, _dst.<k> a link's target. Scoping: a bare name is the CURRENT row; ^name one scope
620out; ^docs ^blocks ^nodes ^links the repository from a row. An unknown bare name is absent plus a hints line.
621
622NESTED: <receiver> collect|exists|none|count|first|single { [select …] [where …] [order by …] [limit N] } over a
623relation, a list key, entries(x) or ^docs; no consumer = collect, a leading predicate = where: docs { type == "x" }.
624FOLLOW: from docs where <seed> follow _out_links collect { from _dst depth 3 } links out; _in_links { from _doc }
625backlinks; refs(before) a property's documents; $depth per row. SEARCH: text("words") prunes; order by semantic("…") desc.
626
627 from docs where layer == "working"
628 select _title, stage from docs where "process" in tags order by _path limit 5
629 select _type, _text from blocks where _path == "/processes/calcination.md"
630 select _path, _value from nodes where _kind == "md:task" and not checked
631 select _path, _block_id from links where _dst_path == "/processes/calcination.md"
632 from docs where _title.lower().contains("salt") and _updated_at >= "2026-01-01"
633 from docs where _nodes exists { where _kind == "md:task" and not checked }
634 select _path, $depth from docs where _path == "/timeline/review.md" follow refs(before), refs(after)
635 docs count { where layer == "canon" }
636 select _seq, _ts, _hash from revisions where _path == "/processes/calcination.md" order by _seq desc
637 select tags, n: count() from docs flatten tags group by tags order by count() desc
638
639Every error is filter_invalid naming what to change; when the engine can correct the call, data.fix IS the call.
640Chapters: sugar paths targets scoping scalars predicates nested select flatten order follow semantic examples; all.
641"#;
642
643#[cfg(test)]
646mod tests {
647 use super::{
659 QUERY_SYNTAX, QUERY_SYNTAX_CARD, QUERY_SYNTAX_SECTION_NAMES, QUERY_SYNTAX_SECTIONS,
660 query_syntax_section,
661 };
662
663 const TARGETS: [&str; 4] = ["docs", "blocks", "nodes", "links"];
664 const CONSUMERS: [&str; 7] = [
665 "collect", "exists", "none", "count", "first", "single", "distinct",
666 ];
667
668 fn has_top_level_from(q: &str) -> bool {
669 let chars: Vec<char> = q.chars().collect();
670 let (mut depth, mut quote): (i32, Option<char>) = (0, None);
671 for (i, &c) in chars.iter().enumerate() {
672 if let Some(open) = quote {
673 if c == open {
674 quote = None;
675 }
676 continue;
677 }
678 match c {
679 '"' | '\'' => quote = Some(c),
680 '{' => depth += 1,
681 '}' => depth -= 1,
682 _ if depth == 0 && chars[i..].starts_with(&[' ', 'f', 'r', 'o', 'm', ' ']) => {
683 return true;
684 }
685 _ => {}
686 }
687 }
688 false
689 }
690
691 fn has_placeholder(q: &str) -> bool {
693 let chars: Vec<char> = q.chars().collect();
694 for (i, &c) in chars.iter().enumerate() {
695 if c != '<' || !chars.get(i + 1).is_some_and(|n| n.is_ascii_alphabetic()) {
696 continue;
697 }
698 let mut j = i + 1;
699 while j < chars.len()
700 && (chars[j].is_ascii_alphabetic() || matches!(chars[j], '_' | '|' | ' ' | '…'))
701 {
702 j += 1;
703 }
704 if chars.get(j) == Some(&'>') {
705 return true;
706 }
707 }
708 false
709 }
710
711 fn is_prose(q: &str) -> bool {
712 q.contains(". ")
713 || q.contains("; ")
714 || q.ends_with('.')
715 || q.ends_with(':')
716 || q.ends_with(';')
717 }
718
719 fn root_consumer(q: &str) -> bool {
720 let mut it = q.splitn(3, ' ');
721 let (Some(target), Some(next)) = (it.next(), it.next()) else {
722 return false;
723 };
724 TARGETS.contains(&target) && (CONSUMERS.contains(&next) || next.starts_with('{'))
725 }
726
727 fn extract(text: &str) -> (Vec<(usize, String, String)>, Vec<String>) {
729 let (mut queries, mut fragments) = (Vec::new(), Vec::new());
730 let mut section = String::new();
731 for (i, raw) in text.lines().enumerate() {
732 if let Some(h) = raw.strip_prefix("## ") {
733 section = h.trim().to_owned();
734 continue;
735 }
736 if !raw.starts_with(" ") || raw.trim().is_empty() {
737 continue;
738 }
739 let trimmed = raw.trim();
740 if section == "Examples" {
741 queries.push((i + 1, section.clone(), trimmed.to_owned()));
742 continue;
743 }
744 let query = trimmed.split(" ").next().unwrap_or("").trim();
745 if has_placeholder(query) || query.contains('…') || is_prose(query) {
746 continue;
747 }
748 if !(query.starts_with("select ") || query.starts_with("from ") || root_consumer(query))
749 {
750 continue;
751 }
752 if query.starts_with("select ") && !has_top_level_from(query) {
753 fragments.push(query.to_owned());
754 continue;
755 }
756 queries.push((i + 1, section.clone(), query.to_owned()));
757 }
758 (queries, fragments)
759 }
760
761 #[test]
762 fn extraction_is_substantial() {
763 let (queries, fragments) = extract(QUERY_SYNTAX);
764 let in_examples = queries.iter().filter(|q| q.1 == "Examples").count();
765 assert!(in_examples >= 20, "{in_examples} examples");
766 assert!(
767 queries.len() - in_examples >= 10,
768 "{} outside Examples",
769 queries.len() - in_examples
770 );
771 for f in &fragments {
772 assert!(
773 !TARGETS.iter().any(|t| f.contains(&format!(" from {t}"))),
774 "a fragment must not carry a top-level from: {f}"
775 );
776 }
777 }
778
779 #[test]
780 fn every_example_parses() {
781 let (queries, _) = extract(QUERY_SYNTAX);
782 let failures: Vec<String> = queries
783 .iter()
784 .filter_map(|(line, section, q)| {
785 oqx::parse_string(q)
786 .err()
787 .map(|e| format!("line {line} ({section}): {q}\n {e}"))
788 })
789 .collect();
790 assert!(
791 failures.is_empty(),
792 "query_syntax examples that do not parse:\n{}",
793 failures.join("\n")
794 );
795 }
796
797 #[test]
798 fn error_examples_live_in_prose() {
799 let (queries, _) = extract(QUERY_SYNTAX);
800 for bad in [
801 "era > 1600 from docs",
802 "from docs count",
803 "from docs where \"pricing\" in list(tags) select layer, tags",
804 ] {
805 assert!(oqx::parse_string(bad).is_err());
806 assert!(
807 !queries.iter().any(|q| q.2 == bad),
808 "{bad} is an error example and must stay in prose"
809 );
810 }
811 }
812
813 #[test]
814 fn clause_order_sentence_matches_the_parser() {
815 let e = oqx::parse_string("from docs where x select y")
816 .unwrap_err()
817 .to_string();
818 let (_, rest) = e
819 .split_once("OQX clause order is ")
820 .expect("the parser names the order");
821 let order = rest
822 .split(" (")
823 .next()
824 .unwrap()
825 .trim_end_matches(['.', ' ']);
826 assert!(
827 QUERY_SYNTAX.contains(&format!("CLAUSE ORDER IS FIXED: {order}")),
828 "the text's order sentence drifted from the parser's ({order})"
829 );
830 }
831
832 fn reference_ts() -> Option<String> {
834 let path = concat!(
835 env!("CARGO_MANIFEST_DIR"),
836 "/../../packages/core/src/mcp/reference.ts"
837 );
838 std::fs::read_to_string(path).ok()
839 }
840
841 fn ts_literal(ts: &str, name: &str) -> String {
845 let open = format!("export const {name} = `");
846 let start = ts.find(&open).expect("the TS export") + open.len();
847 let end = start + ts[start..].find("\n`;").expect("the TS literal's end") + 1;
848 let mut text = String::new();
849 let mut chars = ts[start..end].chars();
850 while let Some(c) = chars.next() {
851 if c == '\\' {
852 if let Some(n) = chars.next() {
853 text.push(n);
854 }
855 } else {
856 text.push(c);
857 }
858 }
859 text
860 }
861
862 #[test]
865 fn twin_is_byte_identical_to_the_reference() {
866 let Some(ts) = reference_ts() else {
867 return;
868 };
869 assert!(
870 ts_literal(&ts, "QUERY_SYNTAX") == QUERY_SYNTAX,
871 "crates/omgbase-surface/src/reference.rs drifted from packages/core/src/mcp/reference.ts"
872 );
873 }
874
875 #[test]
881 fn card_and_sections_are_byte_identical_to_the_reference() {
882 let Some(ts) = reference_ts() else {
883 return;
884 };
885 assert!(
886 ts_literal(&ts, "QUERY_SYNTAX_CARD") == QUERY_SYNTAX_CARD,
887 "QUERY_SYNTAX_CARD drifted from packages/core/src/mcp/reference.ts"
888 );
889 let table_at = ts
890 .find("export const QUERY_SYNTAX_SECTIONS")
891 .expect("the TS section table");
892 let mut cursor = table_at;
893 for (slug, heading) in QUERY_SYNTAX_SECTIONS {
894 let pair = format!("{{ slug: \"{slug}\", heading: \"{heading}\" }}");
895 let at = ts[cursor..]
896 .find(&pair)
897 .unwrap_or_else(|| panic!("the TS table lacks {pair} after the previous pair"));
898 cursor += at + pair.len();
899 }
900 let pairs = ts[table_at..cursor].matches("{ slug: \"").count();
901 assert_eq!(
902 pairs,
903 QUERY_SYNTAX_SECTIONS.len(),
904 "the TS table has pairs this twin lacks"
905 );
906 assert!(
907 !ts[cursor..ts[cursor..].find("];").unwrap() + cursor].contains("{ slug: \""),
908 "the TS table has pairs past this twin's last"
909 );
910 }
911
912 #[test]
917 fn sections_are_the_headings_in_order_and_partition_the_manual() {
918 let headings: Vec<&str> = QUERY_SYNTAX
919 .lines()
920 .filter_map(|l| l.strip_prefix("## "))
921 .collect();
922 let table: Vec<&str> = QUERY_SYNTAX_SECTIONS.iter().map(|(_, h)| *h).collect();
923 assert_eq!(table, headings);
924 let mut names: Vec<&str> = QUERY_SYNTAX_SECTIONS.iter().map(|(s, _)| *s).collect();
925 names.push("all");
926 assert_eq!(names, QUERY_SYNTAX_SECTION_NAMES);
927 let mut joined = Vec::new();
928 for (slug, heading) in QUERY_SYNTAX_SECTIONS {
929 assert!(slug.chars().all(|c| c.is_ascii_lowercase()), "{slug}");
930 let text = query_syntax_section(slug).expect(slug);
931 assert!(text.starts_with(&format!("## {heading}\n")), "{slug}");
932 assert!(text.ends_with('\n') && !text.ends_with("\n\n"), "{slug}");
933 assert!(!text.contains("\n## "), "{slug} holds a second heading");
934 joined.push(text);
935 }
936 let body = &QUERY_SYNTAX[QUERY_SYNTAX.find("\n## ").unwrap() + 1..];
937 let squash = |s: &str| {
938 let mut out = String::new();
939 for l in s.split('\n').filter(|l| !l.is_empty()) {
940 out.push_str(l);
941 out.push('\n');
942 }
943 out
944 };
945 assert_eq!(squash(&joined.join("\n")), squash(body));
946 assert_eq!(query_syntax_section("all").as_deref(), Some(QUERY_SYNTAX));
947 assert_eq!(query_syntax_section("nope"), None);
948 assert!(QUERY_SYNTAX_CARD.contains("CLAUSE ORDER IS FIXED: select, from, where"));
949 for (slug, _) in QUERY_SYNTAX_SECTIONS {
950 assert!(QUERY_SYNTAX_CARD.contains(slug), "the card names {slug}");
951 }
952 }
953
954 #[test]
958 fn every_card_example_parses() {
959 let (queries, _) = extract(QUERY_SYNTAX_CARD);
960 assert!(queries.len() >= 10, "{} card examples", queries.len());
961 let failures: Vec<String> = queries
962 .iter()
963 .filter_map(|(line, _, q)| {
964 oqx::parse_string(q)
965 .err()
966 .map(|e| format!("line {line}: {q}\n {e}"))
967 })
968 .collect();
969 assert!(
970 failures.is_empty(),
971 "card examples that do not parse:\n{}",
972 failures.join("\n")
973 );
974 }
975}