Skip to main content

omgbase_surface/
reference.rs

1//! The `query_syntax` text (`spec/surface/README.md` §4: reference prose,
2//! unpinned). A condensed distillation of `docs/query-language.md`. Byte for
3//! byte the reference's `packages/core/src/mcp/reference.ts` — a test below
4//! compares the two when the monorepo is present, and another parses every
5//! example the text shows.
6
7/// The agent-facing OQX syntax reference the `query_syntax` tool returns.
8pub const QUERY_SYNTAX: &str = r##"# query — OQX syntax reference
9
10The `query` tool takes ONE plain OQX string (+ optional `limit`/`cursor`, and
11`bare: true` to drop the automatic `id`/`path` envelope — needs a `select`).
12There is no {from, filter, order} envelope: every concern is a clause of the
13string. Two rules tell the two syntaxes apart: DOT navigation belongs to the
14host object model (`doc.layer`, `section.blocks`, `dst.title`); WHITESPACE
15query directives belong to OQX — a nested query is `<receiver> collect|exists|
16count|first|single { <block> }` (a postfix directive + a `{ … }` block), never a
17method. Its distinctive power is the receiver-constrained nested query that
18correlates to the current row: `from docs where nodes exists { where kind ==
19"md:task" }` returns only the docs that themselves contain a matching node.
20
21  [select <items>] from <target> [where <pred>] [frontier <pred>] [follow <dest> [{ … }], …]
22                   [group by <key>, …] [having <pred>] [order by <expr> [asc|desc], …] [limit N] [offset N]
23  <target> count|exists|none|first|single { <block> }     (scalar/one-row form: a bare target at the root scope)
24
25CLAUSE ORDER IS FIXED: select, from, where, frontier, follow, group by, having, order by, limit, offset —
26each at most once; an out-of-order clause is a parse error naming the order.
27Only `select` may drop its keyword, and only as the first clause
28(`path, era from docs where …`). A top-level predicate needs `where`, because
29`where` follows `from` (`era > 1600 from docs` is the clause-order error); a
30block may LEAD with a predicate (see Sugar), but a predicate after a projection
31still needs the keyword (`{ name where kind == "md:task" }`, not
32`{ name, kind == "md:task" }`); `from docs count` is an error (write
33`docs count { … }`). `where` and later `select` items may use the same body's
34`select` aliases (`select path, old: era < 1000 from docs where old`; an alias
35shadows a same-named field there). `and` / `or` / `not` are the connectives;
36the symbols `&&` / `||` / `!` are accepted synonyms (the words are canonical and
37what `print` writes).
38
39## Sugar (OQX 0.17)
40
41Every form here is shorthand for an explicit directive (the AST is the explicit form):
42  nodes { kind == "md:task" }        ≡ nodes collect { where kind == "md:task" } — a receiver block with
43                                       no consumer is collect; a block whose LEADING expression is a predicate
44                                       (comparison, call, !/is/not, in, literal, (…), a consumer test — anything
45                                       but a bare name, a dotted path or a ^lift) is where-first. Bare names still
46                                       project: nodes { name } ≡ nodes collect { name }; filter a bare field with
47                                       nodes { is checked }. Wherever a value stands (0.20), call arguments included:
48                                       size(children { is checked }); x distinct { … } ≡ x collect distinct { … }. After a
49                                       follow destination too: follow children { depth 2 } is that destination's block.
50  (a, b)                              a tuple (0.20): two or more expressions, concatenated one level flat — a value
51                                       (size((tags, aliases)), "x" in (tags, aliases)) and a receiver ((a, b) { … },
52                                       follow (before, after) { depth 2 }); (a) alone is grouping
53  refs(x)[0]                          ≡ refs(x) first { offset 0 }   positional (integer literal or ${binding});
54                                       out of range ⇒ absent; refs(x)[0].path navigates the row
55  ^docs[path == ^company]             ≡ ^docs first { where path == ^company }   first match or absent
56  ^docs[path == ^company]!            ≡ … single { … } required: exactly one, else filter_invalid
57  title!                              required: title, or an error naming the expression (and the row's $key — its id)
58                                       — never a filter, never a coercion (0!, ""! are values); tightest: a!.b vs a.b!
59  is x / not x                        truthiness of x, and its negation (prefix; `not` ≡ `!`)
60  x is y / x is not y                 identity ($key: a store row's id, else structural) — compares rows, which == does not;
61                                       x is null = absent; for scalars is ≡ ==; comparison precedence, no chaining
62  a and b / a or b                    the connectives (≡ && / ||: same precedence, short-circuit, value: data.title or path coalesces)
63  boss: ^docs[path == ^manager], bossName: boss.title   a select item may use the items to its LEFT
64  Reserved words (never a bare field name): from where select is not and or, true false null.
65  Contextual words (limit order offset follow collect count first distinct desc by in frontier depth) parse as
66  fields in expression position (select o: order from docs).
67
68Results are LEAN hits — { id, path } + whatever `select` projects (or `values`,
69`count`, `exists`, `none`). The envelope keys ARE the attributes: copy a hit's
70`id` straight into `where id == "…"` (no translation); `select id, path` is simply
71the envelope (no duplicate columns); any OTHER projection whose output key would
72be `id` or `path` must be aliased — `select data.path` is refused ("projection
73`path` collides with the hit envelope — alias it (`p: data.path`)"); a row
74surfacing as a value (an empty `collect { }`) is the same `{ id, path }`. Every
75other tool spells a row the same way (`docs_read` → `{ id, path, rev, properties,
76raw }`, `blocks_read` → `{ id, type, text, raw, hash, … }`, `docs_list` → `{ id,
77path, updated_at, blocks }`, a revision → `{ id, path, hash }`): hydrate by id via
78blocks_read/docs_read.
79
80## Paths
81
82Every path the surface returns is `/`-rooted — the form a reference is
83written in (`[x](/projects/oqx.md)`, `before: [/timeline/kickoff.md]`): a
84hit's `path`, an edge's `dst_path`, every tool's paths. Every path it
85accepts tolerates both forms (`/a.md` or `a.md`): tool arguments, `refs(x)`,
86`within(...)`, and a string LITERAL compared with `path`/`dst_path` (bare, or
87through a row handle: `doc.path`) by `==`/`!=` or passed to their
88`.startsWith(...)` (it is rooted first, so `path == "a.md"` and `path == "/a.md"`
89both match). A property holding a reference compares directly — and is never
90rooted: `where ^path in list(after)`, `where customer == ^path`,
91`where data.path == "/x.md"` (the author's `path:` key, as written) — no
92`"/" + ...` glue.
93
94## Targets & field namespaces
95
96THE NAME RULE: `$` belongs to OQX — $it, $key, $depth, $stop, $leaf, $frontier,
97$ordinal are values the LANGUAGE injects into a scope; omgbase adds NO $ names
98(a `$id` in a query is an unknown name, absent). Every engine attribute is a BARE
99snake_case name that WINS on collision — the convention the relations (children,
100blocks, out, in, doc, frontmatter, …) always followed: a bare name that is an
101engine attribute reads the engine's value, never yours. Your data is bare too;
102where a key of yours collides with an engine name, reach it through a source
103handle: `data.<k>` (the merged authored view — frontmatter + inline, exactly the
104read a bare non-colliding key performs), `frontmatter.<k>`, `inline.<k>`. So
105`path` is the engine's /-rooted location and `data.path` your `path:` key;
106`tags` is your tags ∪ the body #hashtags and `data.tags` the authored list alone;
107`title` the computed title (the first H1) and `data.title` your `title:`;
108`entries(data)` lists every authored top-level key. Same value, same name
109everywhere: a hit's `id` is docs_read's `id` and the `id` argument of blocks_read.
110
111  Reserved bare names per kind (engine attributes; handles; self; relations):
112    docs    id path raw hash updated_at title tags format | data frontmatter inline | doc | blocks nodes out_links in_links revisions
113    blocks  id path doc_id raw hash updated_at type text attrs depth ordinal | block doc | children nodes out_links section
114    nodes   id path doc_id block_id kind name value attrs | section doc block | blocks subsections children
115    links   id path doc_id block_id via_id dst_id dst_path uri field dst_kind predicate provenance anchor | doc
116    revisions  id path rev seq doc_id commit ts origin actor reason hash is_current archived | doc
117  A key of yours with any of these names is `data.<k>` (or frontmatter.<k> /
118  inline.<k> by source; attrs.<k> on a block or node).
119  UNKNOWN NAMES: a name that is none of the above AND that no object of the
120  kind in this repository carries is reported before any row is read — on
121  links as a filter_invalid error naming the fields; on docs/blocks/nodes the
122  result gains `hints` (one line per name, nearest match when there is one)
123  and the rows still come back; in a plan/apply/execute selector it is refused.
124  A property some objects carry is known (partial presence is normal).
125
126docs:
127  - bare identifier  = a document PROPERTY (frontmatter + inline `key:: value`,
128                       unioned; nested via dots: meta.owner). YAML/JSON docs
129                       expose the parsed object's keys the same way.
130  - source handles   = data.<k> (merged authored view) / frontmatter.<k> /
131                       inline.<k>; the whole bag as a collection via entries(data)
132                       / entries(frontmatter) / entries(inline) (see entries()).
133  - attributes       = id, path (/-rooted, see Paths), updated_at (ISO-8601,
134                       sorts chronologically), raw (the whole file, frontmatter
135                       included), hash (its sha256, 64 hex), title (the computed
136                       title: the first H1; data.title is the authored key), tags
137                       (authored tags ∪ body #hashtags, deduped, authored first;
138                       data.tags is the authored list), format (the doc's format —
139                       "markdown"/"yaml"/"json"; present on EVERY doc, so
140                       has(format) is always true and `format == "yaml"` is the
141                       filter).
142  - relations        = nodes, blocks, doc.out_links / doc.in_links (a doc's
143                       links as rows; the documents at their other end are
144                       out_links collect distinct { from dst } and
145                       in_links collect distinct { from doc } — there is no
146                       out / in relation), revisions (the document's stored
147                       revisions, oldest first — see revisions below).
148
149blocks:
150  - bare fields      = doc_id (the owning doc), type, text (normalized), ordinal
151                       (0-based among siblings), depth (0 at the top level), and
152                       every attrs key FLATTENED onto the row (bare `checked`
153                       reads attrs.checked; `attrs.<k>` still works; the
154                       structural fields win on collision; absent ⇒ false).
155  - attributes       = id, path (the owning doc's path), raw (the block's raw
156                       source — a container's raw holds its children), hash
157                       (its sha256 — the value update/split expect as
158                       expect.content_hash), updated_at.
159  - reach-through    = doc.<key> / doc.path / doc.id / doc.format = the
160                       CONTAINING doc (doc.id and doc.path cost nothing — they
161                       read the row's own doc_id / joined path, no lookup).
162  - relations        = block.children, block.nodes, block.out_links, section.
163  NOTE: inside a `follow`, $depth/$ordinal are ALWAYS the walk's metadata; a
164  block's own nesting is `depth` / `ordinal` (no $) everywhere.
165
166nodes:
167  - bare fields      = doc_id, block_id (null for a block-less node), kind
168                       (md:task / md:link / md:section / yaml:…), name, value,
169                       + attrs keys flattened (checked, level).
170  - attributes       = id, path.
171  - reach-through    = doc.<key> / doc.id / doc.path, block.type / block.text
172                       / block.id / block.path.
173  - relations        = section.blocks, section.children, section.subsections.
174
175links:  (the authored link graph as rows — one per open link: the relation a link
176        establishes; the md:link NODE is its occurrence in the text and via_id
177        points at it — a frontmatter link has a row and no node)
178  - bare fields      = doc_id (the SOURCE doc), block_id (the source block; null
179                       for a frontmatter edge), via_id, dst_id (the destination
180                       node id: a doc id, an external node id or phantom:<path>),
181                       dst_path (target doc path, /-rooted; NULL when
182                       dangling/external), uri (external URL; NULL otherwise),
183                       predicate ("references"/"embeds"/freeform), provenance
184                       ("link"/"frontmatter"/"inline_field"/…), dst_kind
185                       ("document"/"external"/"collection"), anchor, field (the
186                       frontmatter key / inline field it was authored in).
187  - attributes       = id, path (the SOURCE doc's path); doc.<key> reaches the
188                       source document. There is no from_commit: a query sees
189                       open edges only (history is blocks_history/changes_since).
190  - text()/semantic() are N/A on links.
191
192revisions:  (history as rows — one per stored revision of every document of the
193        repository, deleted documents included: what docs_history lists, queryable)
194  - bare fields      = rev (= id), seq (1, 2, … per document), doc_id, commit,
195                       ts (the commit's time, sorts chronologically), origin
196                       ("observed"/"api"/"import"/"projection"), actor, reason,
197                       hash (the rendered file's sha256 — docs_history's hash),
198                       is_current (true for the document's current revision),
199                       archived (true when the bodies moved to an archive segment:
200                       the row stays, docs_read { at } reads through the segment).
201  - attributes       = id (the revision id, r_…), path (the path the revision
202                       was committed under — a move re-paths only the current
203                       revision, so earlier ones keep the old path). doc.<key>
204                       reaches the document's CURRENT row; absent for a deleted
205                       document, so where doc == null is the revisions of deleted documents.
206  - on docs          = revisions is a relation (oldest first): a document with
207                       more than one version is docs { revisions count { } > 1 }.
208  - text()/semantic() are N/A on revisions. The body as of a revision is
209    docs_read { path, at: "<rev>" } (docs_read_at is the same call).
210
211## Scoping (ADR-015; the root row since surface 2.0): bare names are LOCAL
212
213  name          the CURRENT row only — never falls through to an enclosing row
214                or the repository; an absent field is simply absent.
215  ^name         one scope OUT (the enclosing row's fields/intrinsics/lifts);
216                ^^name two scopes out. `^path` = the enclosing row's path.
217  ^docs ^nodes ^blocks ^links ^revisions
218                THE ROOT ROW (surface 2.0, oqx 0.18): the repository is the root
219                scope's row, one scope out from a top-level row, so ^docs there is
220                the whole documents scan (uncorrelated until you add a ^ predicate);
221                one caret per enclosing block (^^docs from depth two), or the
222                absolute 0^docs from ANY depth. ^id / 0^id = the repository id.
223                ^$it = the root object: ^$it.docs ≡ ^docs, entries(^$it) names the
224                five collections. At the root scope itself a bare target is the scan
225                (docs count { … }, from docs, entries(docs)).
226                ERRORS: a bare `docs` INSIDE a block reads a property of the current
227                row, which has none — refused ("did you mean ^docs"); `^docs` AT THE
228                TOP LEVEL reaches past the root — refused (write the bare `docs`);
229                so is `$repo` (surface < 2.0; the message names the replacement).
230                nodes/blocks on a doc row are that doc's RELATIONS, not the root.
231  $it           the current item itself (a row, or each ELEMENT when the
232                receiver is a list property); ^$it = the enclosing row.
233                $value is NOT an intrinsic (since oqx 0.15): it reads a
234                property literally named $value, absent when there is none.
235  $key          the row's identity (0.20): a store row's id (doc/block/node/edge), an entries(x)
236                entry's key, a follow occurrence's walk identity; project $key: <expr> to rekey the
237                rows a block yields (a destination block's $key: is not a column).
238
239## Scalar semantics (@omgbase/oqx)
240
241  comparisons  ==  !=  <  <=  >  >=    strict & typed (5 == "5" is false); either
242                                       side may be a field (field-vs-field OK).
243  absence      null and a missing field are the same "absent" value; two absents
244               are EQUAL, so `absent != "x"` is TRUE and `absent == null` is TRUE.
245               Relational (< <= > >=) on an absent operand → FALSE. Bare `f` in
246               boolean position uses JS truthiness (absent/false/0/"" ⇒ false;
247               NOTE: "false", [] and {} are truthy). has(f) = explicit presence.
248  boolean      and  or  not    grouping ( … )   — && || ! are accepted synonyms
249  arithmetic   + - * / %    (+ concatenates if either side is a string)
250  membership   x in y       array → typed membership; string → substring;
251                            object → key exists; range → coverage. Any list
252                            property works directly: "pricing" in tags. list(f)
253                            coerces scalar-or-list (absent → []) — use it when a
254                            key is sometimes scalar, sometimes a list.
255  ranges       lo..hi (inclusive)  lo...hi (exclusive hi)  ..hi  lo..
256               era in 800..1680; works over ISO dates. range(prop) reads a
257               range-shaped STRING property as an interval: "2026-01-15" in range(window).
258  strings      x.contains("s") x.startsWith("p") x.endsWith("s") x.matches("^re$")
259               (real regex) x.lower() x.upper() size(x). ALL CASE-SENSITIVE —
260               fold with .lower(): title.lower().contains("aurora").
261               Infix regex: x ~ "re" is x.matches("re"); x ~* "re" is the
262               case-insensitive match; x !~ "re" and x !~* "re" negate them.
263               Unanchored unless the pattern anchors; an absent x is false
264               under ~ and TRUE under !~ (not SQL's NULL).
265  functions    has(f)  size(x)  list(x)  entries(x)  range(s)
266  dates        a date is an ISO string, a duration a NUMBER OF SECONDS; arithmetic
267               is METHODS, never + or - (+ concatenates strings).
268               date("today"|"now"|"yesterday"|"tomorrow"|"sow"|"eow"|"som"|"eom"|
269               "soy"|"eoy") — the host clock, UTC; date("2026-10-15") canonical ISO
270               (absent if unparseable). dur("1h30m") dur("2 days, 3 hours")
271               dur("P1DT2H") dur(90) → seconds (year = 365 d, month = 30 d).
272               d.plus(dur("30 days"))  d.minus(86400)  d.plus("1 month") (calendar:
273               Jan 31 → Feb 28)  a.until(b) → signed seconds  d.year() d.month()
274               d.day() d.weekday() (1 = Mon) d.hour() d.minute() d.second()
275               d.date()  d.format("EEE dd MMM yyyy")  n.dur_text() → "1h 30m"
276               n.dur_iso() → "PT1H30M". Total: a non-date is absent, so
277               `where date(due) < date("today").minus(dur("30 days"))` just
278               filters. A datetime is never < its own date: compare
279               updated_at.date() == date("today"), or < date("tomorrow").
280  literals     "str" 'str' 123 1.5 true false null
281  determinism  no random; the only clock is date("today"/"now"/…), read once per
282               query. Lists/structs are not literals; ternary is not supported.
283               Errors surface as filter_invalid.
284  scalars vs lists  a list-authored key never == a scalar (tags == "x" is false
285               when tags is a list) — use `"x" in tags` / `in list(tags)`.
286
287## Domain predicates (omgbase)
288
289  text("terms")        FTS5 PRUNING predicate (docs match via their blocks, nodes
290                       via the node index); composes anywhere, incl. nested scopes.
291  semantic("phrase")   embedding cosine SCORE (docs/blocks): threshold it
292                       (semantic("x") > 0.6), project it, or `order by semantic("x")
293                       desc` for top-K. Needs a provider that has embedded the repository; else provider_unavailable.
294  blocks target only:
295    under(id_or_heading)      block in the containment subtree / section range
296    under_heading("Launch")   an ancestor section heading contains the text (ci)
297    under_kind(type[, name])  an ancestor block has that type (name: its text
298                              contains / its key equals name)
299    within("path" | "d_..")   containing doc = id, exact path, or path glob (*);
300                              either path form
301    yaml_path("a.b")          YAML docs: the block at that key path
302    json_pointer("#/a/b")     JSON docs: the block at that pointer
303    has_edge("pred"[, target])  an open authored edge with that predicate leaves here
304    has_anchor()              block carries an authored ^ref
305    parent_type() == "x"      parent block's type (must be compared)
306    child_count() > 0         number of direct children (must be compared)
307  refs(x)              the live documents a property's document references name:
308                       x is a string, a list or absent; each "/a/b.md", "a/b.md"
309                       or "d_…" element resolves to that doc's row; dangling and
310                       non-string elements are dropped. Yields docs rows, so it is
311                       a source, a receiver or a follow destination:
312                       follow refs(before), refs(after) walks a timeline both ways.
313                       The reverse direction needs no function: a document's
314                       path is already the reference form, so
315                       ^docs { where ^path in list(after) } is
316                       "the documents whose `after` names me".
317  A HIT IS A STORE ROW: a top-level row that is not a doc/block/node/edge fails
318                       (filter_invalid). `follow before` over a list of paths
319                       reaches the STRINGS — write `follow refs(before)`.
320
321## Nested queries & consumers
322
323  <receiver> collect|exists|none|count|first|single { <block> }
324    receiver = a relation (nodes, blocks, doc.out_links, section.blocks, …), a
325               reach-through row (dst, doc: one row — dst { title }), a
326    list property (tags), entries(x), or a root collection (^docs; 0^docs at depth ≥ 2).
327    block    = [select …] [from E] [where …] [frontier …] [follow …] [order by …] [limit N] [offset N]
328               — the same fixed order as the top level (`from` optional; the
329               receiver supplies the rows). A leading name/alias list is the
330               projection (`select` dropped); a predicate always needs `where`.
331               A `from E` inside a block RE-PROJECTS the rows through a relation
332               (a typed flatMap): `docs collect { from nodes where kind == "md:task" }`
333               starts at docs, projects each through `nodes`, and queries the
334               resulting nodes; a scalar relation (`from doc`) contributes one
335               row; a missing or non-navigable relation is a loud error, never a
336               global scan.
337  exists    ≥1 row      nodes exists { where kind == "md:task" and not checked }
338  none      0 rows      nodes none { where kind == "md:task" and not checked }  ("every task done")
339  count     a number    nodes count { where kind == "md:task" } >= 2
340  collect   rows (default) — in select for nested results; in where with a lift
341  first / single   zero-or-one (single errors on >1) — lookups in select
342  distinct  on any consumer: dedup by projected value — nodes count distinct { select kind }
343  limit/offset  bound the block's rows AFTER where/order/distinct, BEFORE the
344            consumer reduces: nodes exists { offset 1 } = at least two;
345            first { … offset 1 } = the second.
346  lift      `^name:` in a where-collect binds values outward:
347            select path, open from docs where nodes collect { ^open: value where kind == "md:task" and not checked }
348  joins     ^<t> exists { where slug == ^ref }  semi-join; not … exists  anti-join;
349            ^nodes single { where kind == "person" and attrs.id == ^owner_id }  lookup;
350            x in ^keys  membership over a lifted set.
351
352## select — projection
353
354  select a, alias: expr, nested: nodes collect { … }
355  bare key → the attribute when it is one (id, path, title, …), else the property
356  (scalar-authored → scalar; list → array); data.<k> forces the property;
357  a call/arithmetic needs an alias (n: size(tags)) unless in values mode;
358  an output key of id/path other than the attribute itself needs an alias.
359  raw         whole file bytes (docs) — docs_read is cheaper for one doc.
360  distinct    select distinct type   (dedup hits by projected value)
361  values      select <ONE item> values → bare values instead of records:
362                select distinct type values from docs          → values: ["hub","lab-note",…]
363                select path values from docs                   → values: ["/index.md", …]
364                tags: tags collect { $it values }             → a plain array
365                h: nodes first { name values where kind == "md:section" order by first_ordinal }
366  entries(x)  a record as a collection ($key / $it per entry), key order:
367                select fm: entries(data) collect { k: $key, v: $it }
368                where entries(frontmatter) exists { where $key == "era" and $it > 1600 }
369                entries(inline), entries(attrs), a nested map, or a list (index keys)
370  $semantic_score is not a field — project semantic("…") under an alias instead.
371
372## order by, limit/offset, pagination
373
374  order by era desc, path asc      absent values sort LAST (asc); ties → (path, id)
375  limit N / offset N               define the result SET (integer literals);
376                                   the tool's limit/cursor then page WITHIN it
377  cursor                           opaque keyset on (path,id); a custom order by
378                                   disables it (you get the top page + truncated)
379
380## Graph traversal (follow) and links
381
382  The citation graph is the LINK ROWS walked through a destination block:
383  dst is a link's destination document (absent for an external or dangling
384  link — it then contributes no successor), doc its source.
385  from docs where <seed> follow out_links collect { from dst }     docs the seed links TO
386  from docs where <seed> follow in_links collect { from doc }      backlinks
387    follow out_links collect { from dst depth 3 }                  bound (1..8, default 8) — a clause of the block
388    follow out_links collect { from dst where layer == "canon" }   keep only matching successors
389    follow in_links collect { from doc where ^path in list(before) }   ^ = the row being expanded
390                                                   (^^ = the walk's enclosing scope)
391    follow out_links collect { from dst where predicate == "depends_on" }   one predicate only
392    follow out_links collect { from dst }, in_links collect { from doc }
393                                                   several destinations: their union
394                                                   (one step dedups by identity)
395    follow ^docs { where out_links exists { where dst_path == ^^path } }
396                                                   a destination block over the root's
397                                                   documents, re-read per frontier row
398                                                   (^ = that row) = the backlinks walk
399    follow out_links collect { from dst frontier type == "practitioner" }   cut, keeping the frontier row
400    from docs where type == "process" frontier !verified follow out_links collect { from dst }
401                                                   the BODY's frontier applies to the seeds:
402                                                   an unverified seed is kept, not expanded
403    follow distinct out_links collect { from dst } / … { $key: group from dst }   identity control ($key)
404    follow refs(before), ^docs { where ^path in list(after) }    forward over `before`, backward
405                                                   over every doc whose `after` names me
406  Per-occurrence metadata: $depth (seed = 1), $stop (interior|leaf|frontier|
407  depth|cycle), $leaf/$frontier, $ordinal (a deterministic 1..N rank over the
408  walk, ordered by depth then path — `order by $ordinal` for walk order,
409  `where $ordinal <= N` for a deterministic budget cut), $key — usable in
410  select/order by and the top-level post-walk where (a where conjunct that
411  reads one filters the walk's RESULT; the other conjuncts remain the seed
412  predicate), NOT metadata of the candidates a destination block's where reads,
413  nor valid inside a consumer directive. Cyclic graphs are safe: a revisited
414  node is admitted once as a `$stop == "cycle"` occurrence and never
415  re-expanded. The default keeps one occurrence per distinct walk path;
416  `follow distinct` dedups reached rows by identity.
417  Destination blocks (OQX 0.20): a brace after ANY destination opens that
418  destination's block — children { depth 2 }, refs(x) { depth 2 }, out_links
419  collect { from dst depth 2 }, ^docs { where ^path in list(after) } — with the
420  0.17 body rules (where-first, projections) and two clauses of its own after
421  `where`: frontier <pred> (occurrences reached through this destination that
422  match are frontiers; ^ there is the walk's enclosing scope) and depth <n>
423  (this destination expands a frontier row only while $depth < n). The body
424  that carries the follow may write frontier <pred> too, after its where and
425  before follow: it applies to the SEEDS (read like the body's where) — a
426  matching seed stays in the result with $stop == "frontier" and is not
427  expanded; each destination's frontier is its own. There is no options block;
428  the old `by` is $key. A bare follow ^docs (no brace, no consumer) is an
429  error naming both block spellings.
430  Type-preserving relations: block.children, section.children, section.subsections;
431  the citation graph is out_links collect { from dst } / in_links collect { from doc }.
432  from links where dst_path == "/notes/x.md"       inbound links to a doc, as rows
433  select d: dst.title, k: dst_kind from links       the destination document's title
434  from docs where out_links exists { where dst.layer == "draft" }   links into drafts
435  select b: in_links collect distinct { from doc } from docs     backlinks as { id, path }
436  select doc_id, dst_path from links where predicate == "depends_on"
437  from docs where not doc.in_links exists { }      orphans (nothing links in)
438  (The `graph` tool is a convenience wrapper that compiles to a follow query.)
439
440## semantic grain
441
442from docs ranks whole DOCUMENTS (one hit per file); from blocks ranks PASSAGES.
443text()/where prune the candidate set; only order by reweights.
444
445## Examples
446
447  from docs where layer == "working"
448  from docs where path.startsWith("/guides/") and updated_at >= "2026-08-01"
449  select layer, tags from docs where "pricing" in list(tags)
450  from docs where inline.owner == "alice"
451  from docs where title.lower().contains("q3 plan")
452  from docs where format == "yaml"
453  from docs where type == "practitioner" order by era desc limit 2
454  from docs where nodes none { where kind == "md:task" and not checked }
455  from docs where tags exists { where $it == "tria-prima" }
456  select fm: entries(data) collect { k: $key, v: $it } from docs where path == "/x.md"
457  select owner_id, owner: ^nodes single { where kind == "person" and attrs.id == ^owner_id } from docs
458  from blocks where type == "task" and not checked and under_heading("Launch") and doc.layer == "working"
459  from blocks where type == "paragraph" and has_edge("references", "d_92aaaaa")
460  select path, $depth from docs where path == "/timeline/review.md" follow refs(before), refs(after)
461  select path, next: ^docs collect { path where ^path in list(before) } from docs where type == "milestone"
462  select path, n: size(^docs), repo: ^id from docs limit 1
463  select p: data.path, t: data.title from docs where has(data.path)
464  docs { type == "substance" and not verified }
465  select subject, process: ^docs[slug == ^subject]!.path from docs where type == "lab-note"
466  select path, back: nodes first { select p: 0^docs collect { path where ^^path in list(after) } values } from docs where type == "milestone"
467  select path, prior: refs(before) collect { path, when } from docs where type == "milestone"
468  from blocks order by semantic("identity preservation across edits") desc limit 10
469  select uri from links where dst_kind == "external"
470  docs count { where layer == "canon" }
471  select seq, origin, ts, hash from revisions where path == "/processes/calcination.md" order by seq desc
472  from docs where revisions count { where origin == "api" } > 0
473  select p: path, d: doc.path from revisions where doc == null
474  select layer, n: count(), titles: collect(title), open: rows count { where nodes exists { where kind == "md:task" and not checked } } from docs group by layer having count() > 1 order by count() desc
475
476Common mistake: `$path`/`$id` (the 2.x spelling) — omgbase has no $ names; write
477`path`/`id`. Another: `where tags == "x"` on a list-valued key is false — write
478`"x" in tags`. Another: `select data.path` — alias it (`p: data.path`).
479"##;
480
481#[cfg(test)]
482mod tests {
483    //! Every OQX example the text shows must parse — extracted MECHANICALLY by
484    //! the rule the reference's `packages/core/src/mcp/reference.test.ts` states
485    //! (and mirrors here): (1) only indented lines are candidates — an error
486    //! example belongs in prose, a code line is a valid query; (2) in
487    //! `## Examples` every indented line is a whole query; (3) elsewhere the
488    //! query is the text before the first run of three or more spaces, skipped
489    //! when it holds a grammar placeholder (`<pred>`), an ellipsis (`…`) or
490    //! sentence punctuation (wrapped commentary); (4) a candidate is a query
491    //! when it begins `select `, `from `, or a root target followed by a
492    //! consumer or `{` (a `^`-led line is an expression fragment); (5) a
493    //! `select`-led candidate without a depth-0 ` from ` is a fragment.
494    use super::QUERY_SYNTAX;
495
496    const TARGETS: [&str; 4] = ["docs", "blocks", "nodes", "links"];
497    const CONSUMERS: [&str; 7] = [
498        "collect", "exists", "none", "count", "first", "single", "distinct",
499    ];
500
501    fn has_top_level_from(q: &str) -> bool {
502        let chars: Vec<char> = q.chars().collect();
503        let (mut depth, mut quote): (i32, Option<char>) = (0, None);
504        for (i, &c) in chars.iter().enumerate() {
505            if let Some(open) = quote {
506                if c == open {
507                    quote = None;
508                }
509                continue;
510            }
511            match c {
512                '"' | '\'' => quote = Some(c),
513                '{' => depth += 1,
514                '}' => depth -= 1,
515                _ if depth == 0 && chars[i..].starts_with(&[' ', 'f', 'r', 'o', 'm', ' ']) => {
516                    return true;
517                }
518                _ => {}
519            }
520        }
521        false
522    }
523
524    /// `<seed>`, `<pred>`, `<ONE item>`: a `<` + letter … `>` run of letters, `_`, `|`, space or `…`.
525    fn has_placeholder(q: &str) -> bool {
526        let chars: Vec<char> = q.chars().collect();
527        for (i, &c) in chars.iter().enumerate() {
528            if c != '<' || !chars.get(i + 1).is_some_and(|n| n.is_ascii_alphabetic()) {
529                continue;
530            }
531            let mut j = i + 1;
532            while j < chars.len()
533                && (chars[j].is_ascii_alphabetic() || matches!(chars[j], '_' | '|' | ' ' | '…'))
534            {
535                j += 1;
536            }
537            if chars.get(j) == Some(&'>') {
538                return true;
539            }
540        }
541        false
542    }
543
544    fn is_prose(q: &str) -> bool {
545        q.contains(". ")
546            || q.contains("; ")
547            || q.ends_with('.')
548            || q.ends_with(':')
549            || q.ends_with(';')
550    }
551
552    fn root_consumer(q: &str) -> bool {
553        let mut it = q.splitn(3, ' ');
554        let (Some(target), Some(next)) = (it.next(), it.next()) else {
555            return false;
556        };
557        TARGETS.contains(&target) && (CONSUMERS.contains(&next) || next.starts_with('{'))
558    }
559
560    /// `(line, section, query)` for every admitted example, plus the skipped `select` fragments.
561    fn extract(text: &str) -> (Vec<(usize, String, String)>, Vec<String>) {
562        let (mut queries, mut fragments) = (Vec::new(), Vec::new());
563        let mut section = String::new();
564        for (i, raw) in text.lines().enumerate() {
565            if let Some(h) = raw.strip_prefix("## ") {
566                section = h.trim().to_owned();
567                continue;
568            }
569            if !raw.starts_with("  ") || raw.trim().is_empty() {
570                continue;
571            }
572            let trimmed = raw.trim();
573            if section == "Examples" {
574                queries.push((i + 1, section.clone(), trimmed.to_owned()));
575                continue;
576            }
577            let query = trimmed.split("   ").next().unwrap_or("").trim();
578            if has_placeholder(query) || query.contains('…') || is_prose(query) {
579                continue;
580            }
581            if !(query.starts_with("select ") || query.starts_with("from ") || root_consumer(query))
582            {
583                continue;
584            }
585            if query.starts_with("select ") && !has_top_level_from(query) {
586                fragments.push(query.to_owned());
587                continue;
588            }
589            queries.push((i + 1, section.clone(), query.to_owned()));
590        }
591        (queries, fragments)
592    }
593
594    #[test]
595    fn extraction_is_substantial() {
596        let (queries, fragments) = extract(QUERY_SYNTAX);
597        let in_examples = queries.iter().filter(|q| q.1 == "Examples").count();
598        assert!(in_examples >= 20, "{in_examples} examples");
599        assert!(
600            queries.len() - in_examples >= 10,
601            "{} outside Examples",
602            queries.len() - in_examples
603        );
604        for f in &fragments {
605            assert!(
606                !TARGETS.iter().any(|t| f.contains(&format!(" from {t}"))),
607                "a fragment must not carry a top-level from: {f}"
608            );
609        }
610    }
611
612    #[test]
613    fn every_example_parses() {
614        let (queries, _) = extract(QUERY_SYNTAX);
615        let failures: Vec<String> = queries
616            .iter()
617            .filter_map(|(line, section, q)| {
618                oqx::parse_string(q)
619                    .err()
620                    .map(|e| format!("line {line} ({section}): {q}\n    {e}"))
621            })
622            .collect();
623        assert!(
624            failures.is_empty(),
625            "query_syntax examples that do not parse:\n{}",
626            failures.join("\n")
627        );
628    }
629
630    #[test]
631    fn error_examples_live_in_prose() {
632        let (queries, _) = extract(QUERY_SYNTAX);
633        for bad in [
634            "era > 1600 from docs",
635            "from docs count",
636            "from docs where \"pricing\" in list(tags) select layer, tags",
637        ] {
638            assert!(oqx::parse_string(bad).is_err());
639            assert!(
640                !queries.iter().any(|q| q.2 == bad),
641                "{bad} is an error example and must stay in prose"
642            );
643        }
644    }
645
646    #[test]
647    fn clause_order_sentence_matches_the_parser() {
648        let e = oqx::parse_string("from docs where x select y")
649            .unwrap_err()
650            .to_string();
651        let (_, rest) = e
652            .split_once("OQX clause order is ")
653            .expect("the parser names the order");
654        let order = rest
655            .split(" (")
656            .next()
657            .unwrap()
658            .trim_end_matches(['.', ' ']);
659        assert!(
660            QUERY_SYNTAX.contains(&format!("CLAUSE ORDER IS FIXED: {order}")),
661            "the text's order sentence drifted from the parser's ({order})"
662        );
663    }
664
665    /// The twin is byte for byte the reference's `QUERY_SYNTAX` (the TS
666    /// template literal, unescaped). Skipped outside the monorepo.
667    #[test]
668    fn twin_is_byte_identical_to_the_reference() {
669        let path = concat!(
670            env!("CARGO_MANIFEST_DIR"),
671            "/../../packages/core/src/mcp/reference.ts"
672        );
673        let Ok(ts) = std::fs::read_to_string(path) else {
674            return;
675        };
676        let open = "export const QUERY_SYNTAX = `";
677        let start = ts.find(open).expect("the TS export") + open.len();
678        let end = ts.rfind("`;").expect("the TS literal's end");
679        let mut text = String::new();
680        let mut chars = ts[start..end].chars();
681        while let Some(c) = chars.next() {
682            if c == '\\' {
683                if let Some(n) = chars.next() {
684                    text.push(n);
685                }
686            } else {
687                text.push(c);
688            }
689        }
690        assert!(
691            text == QUERY_SYNTAX,
692            "crates/omgbase-surface/src/reference.rs drifted from packages/core/src/mcp/reference.ts"
693        );
694    }
695}