pub const QUERY_SYNTAX: &str = r##"# query — OQX syntax reference
The `query` tool takes ONE plain OQX string (+ optional `limit`/`cursor`, and
`bare: true` to drop the automatic `_id`/`_path` envelope — needs a `select`).
There is no {from, filter, order} envelope: every concern is a clause of the
string. Two rules tell the two syntaxes apart: DOT navigation belongs to the
host object model (`_doc.layer`, `_section._blocks`, `_dst._title`); WHITESPACE
query directives belong to OQX — a nested query is `<receiver> collect|exists|
count|first|single { <block> }` (a postfix directive + a `{ … }` block), never a
method. Its distinctive power is the receiver-constrained nested query that
correlates to the current row: `from docs where _nodes exists { where _kind ==
"md:task" }` returns only the docs that themselves contain a matching node.
[select <items>] from <target> [where <pred>] [frontier <pred>] [follow <dest> [{ … }], …]
[flatten <expr> [{ … }] [as <name>]]… [group by <key>, …] [having <pred>]
[order by <expr> [asc|desc], …] [limit N] [offset N]
<target> count|exists|none|first|single { <block> } (scalar/one-row form: a bare target at the root scope)
CLAUSE ORDER IS FIXED: select, from, where, frontier, follow, flatten, group by, having, order by, limit, offset —
each at most once (`flatten` may repeat); an out-of-order clause is a parse error naming the order.
Only `select` may drop its keyword, and only as the first clause
(`_path, era from docs where …`). A top-level predicate needs `where`, because
`where` follows `from` (`era > 1600 from docs` is the clause-order error); a
block may LEAD with a predicate (see Sugar), but a predicate after a projection
still needs the keyword (`{ _name where _kind == "md:task" }`, not
`{ _name, _kind == "md:task" }`); `from docs count` is an error (write
`docs count { … }`). `where` and later `select` items may use the same body's
`select` aliases (`select _path, old: era < 1000 from docs where old`; an alias
shadows a same-named field there). `and` / `or` / `not` are the connectives;
the symbols `&&` / `||` / `!` are accepted synonyms (the words are canonical and
what `print` writes).
## Sugar (OQX 0.17)
Every form here is shorthand for an explicit directive (the AST is the explicit form):
nodes { _kind == "md:task" } ≡ nodes collect { where _kind == "md:task" } — a receiver block with
no consumer is collect; a block whose LEADING expression is a predicate
(comparison, call, !/is/not, in, literal, (…), a consumer test — anything
but a bare name, a dotted path or a ^lift) is where-first. Names still
project: _nodes { _name } ≡ _nodes collect { _name }; filter a bare field with
_nodes { is checked }. Wherever a value stands (0.20), call arguments included:
size(_children { is checked }); x distinct { … } ≡ x collect distinct { … }. After a
follow destination too: follow _children { depth 2 } is that destination's block.
(a, b) a tuple (0.20): two or more expressions, concatenated one level flat — a value
(size((tags, aliases)), "x" in (tags, aliases)) and a receiver ((a, b) { … },
follow (before, after) { depth 2 }); (a) alone is grouping
refs(x)[0] ≡ refs(x) first { offset 0 } positional (integer literal or ${binding});
out of range ⇒ absent; refs(x)[0]._path navigates the row
^docs[_path == ^company] ≡ ^docs first { where _path == ^company } first match or absent
^docs[_path == ^company]! ≡ … single { … } required: exactly one, else filter_invalid
_title! required: _title, or an error naming the expression (and the row's $key — its _id)
— never a filter, never a coercion (0!, ""! are values); tightest: a!.b vs a.b!
is x / not x truthiness of x, and its negation (prefix; `not` ≡ `!`)
x is y / x is not y identity ($key: a store row's id, else structural) — compares rows, which == does not;
x is null = absent; for scalars is ≡ ==; comparison precedence, no chaining
a and b / a or b the connectives (≡ && / ||: same precedence, short-circuit, value: title or _title coalesces)
boss: ^docs[_path == ^manager], bossName: boss._title a select item may use the items to its LEFT
Reserved words (never a bare field name): from where select is not and or, true false null.
Contextual words (limit order offset follow collect count first distinct desc by in frontier depth group having
flatten as) parse as fields in expression position (select o: order from docs).
Results are LEAN hits — { _id, _path } + whatever `select` projects (or `values`,
`count`, `exists`, `none`). The envelope keys ARE the attributes: copy a hit's
`_id` straight into `where _id == "…"` (no translation); `select _id, _path` is
simply the envelope (no duplicate columns); an alias `_id:`/`_path:` on any OTHER
expression is refused — `select _path: _doc._path from blocks` fails ("projection
`_path` collides with the hit envelope — alias it (`p: _doc._path`)"); a row
surfacing as a value (an empty `collect { }`) is the same `{ _id, _path }`. Every
other tool spells a row the same way (`docs_read` → `{ _id, _path, rev, properties,
_raw }`, `blocks_read` → `{ _id, _type, _text, _raw, _hash, … }`, `docs_list` → `{ _id,
_path, _updated_at, blocks }`, a revision → `{ _id, _path, _hash }`): hydrate by `_id`
via blocks_read/docs_read.
## Paths
Every path the surface returns is `/`-rooted — the form a reference is
written in (`[x](/projects/oqx.md)`, `before: [/timeline/kickoff.md]`): a
hit's `_path`, an edge's `_dst_path`, every tool's paths. Every path it
accepts tolerates both forms (`/a.md` or `a.md`): tool arguments, `refs(x)`,
`within(...)`, and a string LITERAL compared with `_path`/`_dst_path` (on the row,
or through a row handle: `_doc._path`, `_dst._path`, `$it._path`) by `==`/`!=` or
passed to their `.startsWith(...)` (it is rooted first, so `_path == "a.md"` and
`_path == "/a.md"` both match). A property holding a reference compares directly —
and is never rooted: `where ^_path in list(after)`, `where customer == ^_path`,
`where path == "/x.md"` (YOUR authored `path:` key — the bare name — compared as
written) — no `"/" + ...` glue.
## Targets & field namespaces
THE NAME RULE — one rule on every record kind: a BARE name is YOURS (an authored
property: frontmatter + inline on a doc, an attrs key on a block or node), an
UNDERSCORE-prefixed name is the ENGINE's (however derived: `_id`, `_path`,
`_title`, `_updated_at`, `_doc`), and `$` belongs to OQX — $it, $key, $depth,
$stop, $leaf, $frontier, $ordinal are values the LANGUAGE injects into a scope;
omgbase adds NO $ names (a `$id` in a query is an unknown name, absent). Nothing
bare is reserved: `title`, `tags`, `path`, `id`, `format`, `doc` are your keys
when you wrote them. Collections stay bare (`from docs`, `^docs`, `docs count { … }`,
`entries(^$it)`) — `from` names a collection, not a row property — and so do the
row functions in call position (`text("x")`, `under(…)`, `refs(x)`): a call is
never an entry read. There is NO self alias (the 4.x `doc`/`block`/`section`):
`$it` is the self, and `_doc`/`_block`/`_dst` reach ANOTHER row. So `_path` is the
engine's /-rooted location and `path` your `path:` key; `_tags` is your tags ∪ the
body #hashtags and `tags` the authored list alone; `_title` the computed title (the
first H1) and `title` your `title:`; `_data.<k>` ≡ `<k>` (the merged authored view —
kept for `entries(_data)` and to reach an authored key that itself starts with
`_`), `_frontmatter.<k>` / `_inline.<k>` one source each. Same value, same name
everywhere: a hit's `_id` is docs_read's `_id` and the `_id` argument of blocks_read.
Engine names per kind (attributes | handles | relations):
docs _id _path _raw _hash _updated_at _created_at _title _tags _format | _data _frontmatter _inline | _blocks _nodes _out_links _in_links _revisions
blocks _id _path _doc_id _raw _hash _updated_at _created_at _type _text _attrs _depth _ordinal | _doc | _children _nodes _out_links _section
nodes _id _path _doc_id _block_id _kind _name _value _attrs | _doc _block | _blocks _subsections _children
links _id _path _doc_id _block_id _via_id _dst_id _dst_path _uri _field _dst_kind _predicate _provenance _anchor | _doc _dst _block | —
revisions _id _path _rev _seq _doc_id _commit _ts _origin _actor _reason _hash _is_current _archived | _doc | —
root _id (^_id, 0^_id) | — | docs blocks nodes links revisions
Every bare name is yours; `_attrs.<k>` on a block or node is the same read as
the bare attrs key `<k>`.
UNKNOWN NAMES: a bare name that no object of the kind in this repository
carries is a property read that yields absent — on docs/blocks/nodes the result
gains `hints` (one line per name, the nearest match drawn from your keys and the
engine names: "no document in this repository carries a property `title`; did
you mean `_title`?") and the rows still come back; in a plan/apply/execute
selector it is refused. On links and revisions (closed: no authored data reaches
them) a bare name is a filter_invalid error. A `_x` that is no engine name of
the kind is filter_invalid naming the kind's engine names. A property some
objects carry is known (partial presence is normal).
docs:
- bare identifier = a document PROPERTY (frontmatter + inline `key:: value`,
unioned; nested via dots: meta.owner). YAML/JSON docs
expose the parsed object's keys the same way. Nothing is
reserved: `title`, `tags`, `path`, `id` are your keys.
- source handles = _data.<k> (the merged authored view — the same read as the
bare key) / _frontmatter.<k> / _inline.<k>; the whole bag
as a collection via entries(_data) / entries(_frontmatter)
/ entries(_inline) (see entries()).
- attributes = _id, _path (/-rooted, see Paths), _updated_at (ISO-8601,
sorts chronologically), _created_at, _raw (the whole file,
frontmatter included), _hash (its sha256, 64 hex), _title
(the computed title: the first H1; `title` is the authored
key), _tags (authored tags ∪ body #hashtags, deduped,
authored first; `tags` is the authored list), _format (the
doc's format — "markdown"/"yaml"/"json"; present on EVERY
doc, so has(_format) is always true and `_format == "yaml"`
is the filter).
- relations = _nodes, _blocks, _out_links / _in_links (a doc's links as
rows; the documents at their other end are
_out_links collect distinct { from _dst } and
_in_links collect distinct { from _doc } — there is no
out / in relation), _revisions (the document's stored
revisions, oldest first — see revisions below).
blocks:
- bare fields = every attrs key FLATTENED onto the row (bare `checked`;
`_attrs.<k>` is the same read; absent ⇒ false) — a block's
authored side.
- attributes = _id, _path (the owning doc's path), _doc_id (the owning
doc), _type, _text (normalized), _ordinal (0-based among
siblings), _depth (0 at the top level), _raw (the block's
raw source — a container's raw holds its children), _hash
(its sha256 — the value update/split expect as
expect.content_hash), _updated_at, _created_at, _attrs.
- reach-through = _doc.<key> / _doc._path / _doc._id / _doc._format = the
CONTAINING doc (_doc._id and _doc._path cost nothing — they
read the row's own _doc_id / joined path, no lookup).
- relations = _children, _nodes, _out_links, _section (no self alias:
the 4.x `block.children` is `_children`).
NOTE: inside a `follow`, $depth/$ordinal are ALWAYS the walk's metadata; a
block's own nesting is `_depth` / `_ordinal` everywhere.
nodes:
- bare fields = the attrs keys flattened onto the row (checked, level,
first_ordinal; `_attrs.<k>` is the same read).
- attributes = _id, _path, _doc_id, _block_id (null for a block-less
node), _kind (md:task / md:link / md:section / yaml:…),
_name, _value, _attrs.
- reach-through = _doc.<key> / _doc._id / _doc._path, _block._type /
_block._text / _block._id / _block._path.
- relations = _blocks, _children, _subsections (a section node's; the
4.x `section.blocks` is `_blocks`).
links: (the authored link graph as rows — one per open link: the relation a link
establishes; the md:link NODE is its occurrence in the text and _via_id
points at it — a frontmatter link has a row and no node)
- attributes = _id, _path (the SOURCE doc's path), _doc_id (the SOURCE
doc), _block_id (the source block; null for a frontmatter
edge), _via_id, _dst_id (the destination node id: a doc id,
an external node id or phantom:<path>), _dst_path (target
doc path, /-rooted; NULL when dangling/external), _uri
(external URL; NULL otherwise), _predicate
("references"/"embeds"/freeform), _provenance
("link"/"frontmatter"/"inline_field"/…), _dst_kind
("document"/"external"/"collection"), _anchor, _field (the
frontmatter key / inline field it was authored in).
- reach-through = _doc.<key> (the source document), _dst.<key> (the
destination document; absent when dangling/external),
_block.<key> (the source block). Closed: links carry no
authored data, so a bare name is filter_invalid. There is
no from_commit: a query sees open edges only (history is
blocks_history/changes_since).
- text()/semantic() are N/A on links.
revisions: (history as rows — one per stored revision of every document of the
repository, deleted documents included: what docs_history lists, queryable)
- attributes = _id (the revision id, r_…), _rev (= _id), _seq (1, 2, … per
document), _doc_id, _commit, _ts (the commit's time, sorts
chronologically), _origin ("observed"/"api"/"import"/
"projection"), _actor, _reason, _hash (the rendered file's
sha256 — docs_history's _hash), _is_current (true for the
document's current revision), _archived (true when the
bodies moved to an archive segment: the row stays,
docs_read { at } reads through the segment), _path (the
path the revision was committed under — a move re-paths
only the current revision, so earlier ones keep the old
path).
- reach-through = _doc.<key> reaches the document's CURRENT row; absent for
a deleted document, so where _doc == null is the revisions
of deleted documents. Closed: a bare name is filter_invalid.
- on docs = _revisions is a relation (oldest first): a document with
more than one version is docs { _revisions count { } > 1 }.
- text()/semantic() are N/A on revisions. The body as of a revision is
docs_read { _path, at: "<rev>" } (docs_read_at is the same call).
## Scoping (ADR-015; the root row since surface 2.0): bare names are LOCAL
name the CURRENT row only — never falls through to an enclosing row
or the repository; an absent field is simply absent.
^name one scope OUT (the enclosing row's fields/intrinsics/lifts);
^^name two scopes out. `^_path` = the enclosing row's path.
^docs ^nodes ^blocks ^links ^revisions
THE ROOT ROW (surface 2.0, oqx 0.18): the repository is the root
scope's row, one scope out from a top-level row, so ^docs there is
the whole documents scan (uncorrelated until you add a ^ predicate);
one caret per enclosing block (^^docs from depth two), or the
absolute 0^docs from ANY depth. ^_id / 0^_id = the repository id.
^$it = the root object: ^$it.docs ≡ ^docs, entries(^$it) names the
five collections. At the root scope itself a bare target is the scan
(docs count { … }, from docs, entries(docs)).
ERRORS: a bare `docs` INSIDE a block reads a property of the current
row, which has none — refused ("did you mean ^docs"); so is a bare
`blocks`/`nodes`/`revisions` there (a doc's own relations are
`_blocks`/`_nodes`/`_revisions`); `^docs` AT THE TOP LEVEL reaches
past the root — refused (write the bare `docs`); so is `$repo`
(surface < 2.0; the message names the replacement).
$it the current item itself (a row, or each ELEMENT when the
receiver is a list property); ^$it = the enclosing row, ^$it._id
its id. $value is NOT an intrinsic (since oqx 0.15): it reads a
property literally named $value, absent when there is none.
$key the row's identity (0.20): a store row's id (doc/block/node/edge), an entries(x)
entry's key, a follow occurrence's walk identity; project $key: <expr> to rekey the
rows a block yields (a destination block's $key: is not a column).
## Scalar semantics (@omgbase/oqx)
comparisons == != < <= > >= strict & typed (5 == "5" is false); either
side may be a field (field-vs-field OK).
absence null and a missing field are the same "absent" value; two absents
are EQUAL, so `absent != "x"` is TRUE and `absent == null` is TRUE.
Relational (< <= > >=) on an absent operand → FALSE. Bare `f` in
boolean position uses JS truthiness (absent/false/0/"" ⇒ false;
NOTE: "false", [] and {} are truthy). has(f) = explicit presence.
boolean and or not grouping ( … ) — && || ! are accepted synonyms
arithmetic + - * / % (+ concatenates if either side is a string)
membership x in y array → typed membership; string → substring;
object → key exists; range → coverage. Any list
property works directly: "pricing" in tags. list(f)
coerces scalar-or-list (absent → []) — use it when a
key is sometimes scalar, sometimes a list.
ranges lo..hi (inclusive) lo...hi (exclusive hi) ..hi lo..
era in 800..1680; works over ISO dates. range(prop) reads a
range-shaped STRING property as an interval: "2026-01-15" in range(window).
strings x.contains("s") x.startsWith("p") x.endsWith("s") x.matches("^re$")
(real regex) x.lower() x.upper() size(x). ALL CASE-SENSITIVE —
fold with .lower(): _title.lower().contains("aurora").
Infix regex: x ~ "re" is x.matches("re"); x ~* "re" is the
case-insensitive match; x !~ "re" and x !~* "re" negate them.
Unanchored unless the pattern anchors; an absent x is false
under ~ and TRUE under !~ (not SQL's NULL).
functions has(f) size(x) list(x) entries(x) range(s)
dates a date is an ISO string, a duration a NUMBER OF SECONDS; arithmetic
is METHODS, never + or - (+ concatenates strings).
date("today"|"now"|"yesterday"|"tomorrow"|"sow"|"eow"|"som"|"eom"|
"soy"|"eoy") — the host clock, UTC; date("2026-10-15") canonical ISO
(absent if unparseable). dur("1h30m") dur("2 days, 3 hours")
dur("P1DT2H") dur(90) → seconds (year = 365 d, month = 30 d).
d.plus(dur("30 days")) d.minus(86400) d.plus("1 month") (calendar:
Jan 31 → Feb 28) a.until(b) → signed seconds d.year() d.month()
d.day() d.weekday() (1 = Mon) d.hour() d.minute() d.second()
d.date() d.format("EEE dd MMM yyyy") n.dur_text() → "1h 30m"
n.dur_iso() → "PT1H30M". Total: a non-date is absent, so
`where date(due) < date("today").minus(dur("30 days"))` just
filters. A datetime is never < its own date: compare
_updated_at.date() == date("today"), or < date("tomorrow").
literals "str" 'str' 123 1.5 true false null
determinism no random; the only clock is date("today"/"now"/…), read once per
query. Lists/structs are not literals; ternary is not supported.
Errors surface as filter_invalid.
scalars vs lists a list-authored key never == a scalar (tags == "x" is false
when tags is a list) — use `"x" in tags` / `in list(tags)`.
## Domain predicates (omgbase)
text("terms") FTS5 PRUNING predicate (docs match via their blocks, nodes
via the node index); composes anywhere, incl. nested scopes.
semantic("phrase") embedding cosine SCORE (docs/blocks): threshold it
(semantic("x") > 0.6), project it, or `order by semantic("x")
desc` for top-K. Needs a provider that has embedded the repository; else provider_unavailable.
blocks target only:
under(id_or_heading) block in the containment subtree / section range
under_heading("Launch") an ancestor section heading contains the text (ci)
under_kind(type[, name]) an ancestor block has that type (name: its text
contains / its key equals name)
within("path" | "d_..") containing doc = id, exact path, or path glob (*);
either path form
yaml_path("a.b") YAML docs: the block at that key path
json_pointer("#/a/b") JSON docs: the block at that pointer
has_edge("pred"[, target]) an open authored edge with that predicate leaves here
has_anchor() block carries an authored ^ref
parent_type() == "x" parent block's type (must be compared)
child_count() > 0 number of direct children (must be compared)
refs(x) the live documents a property's document references name:
x is a string, a list or absent; each "/a/b.md", "a/b.md"
or "d_…" element resolves to that doc's row; dangling and
non-string elements are dropped. Yields docs rows, so it is
a source, a receiver or a follow destination:
follow refs(before), refs(after) walks a timeline both ways.
The reverse direction needs no function: a document's
_path is already the reference form, so
^docs { where ^_path in list(after) } is
"the documents whose `after` names me".
A HIT IS A STORE ROW: a top-level row that is not a doc/block/node/edge fails
(filter_invalid). `follow before` over a list of paths
reaches the STRINGS — write `follow refs(before)`.
## Nested queries & consumers
<receiver> collect|exists|none|count|first|single { <block> }
receiver = a relation (_nodes, _blocks, _doc._out_links, _section._blocks, …), a
reach-through row (_dst, _doc: one row — _dst { _title }), a
list property (tags), entries(x), or a root collection (^docs; 0^docs at depth ≥ 2).
block = [select …] [from E] [where …] [frontier …] [follow …] [flatten …] [group by …] [having …] [order by …] [limit N] [offset N]
— the same fixed order as the top level (`from` optional; the
receiver supplies the rows). A leading name/alias list is the
projection (`select` dropped); a predicate always needs `where`.
A `from E` inside a block RE-PROJECTS the rows through a relation
(a typed flatMap): `docs collect { from _nodes where _kind == "md:task" }`
starts at docs, projects each through `_nodes`, and queries the
resulting nodes; a scalar relation (`from _doc`) contributes one
row; a missing or non-navigable relation is a loud error, never a
global scan.
exists ≥1 row _nodes exists { where _kind == "md:task" and not checked }
none 0 rows _nodes none { where _kind == "md:task" and not checked } ("every task done")
count a number _nodes count { where _kind == "md:task" } >= 2
collect rows (default) — in select for nested results; in where with a lift
first / single zero-or-one (single errors on >1) — lookups in select
distinct on any consumer: dedup by projected value — _nodes count distinct { select _kind }
limit/offset bound the block's rows AFTER where/order/distinct, BEFORE the
consumer reduces: _nodes exists { offset 1 } = at least two;
first { … offset 1 } = the second.
lift `^name:` in a where-collect binds values outward:
select _path, open from docs where _nodes collect { ^open: _value where _kind == "md:task" and not checked }
joins ^<t> exists { where slug == ^ref } semi-join; not … exists anti-join;
^nodes single { where _kind == "person" and _attrs.id == ^owner_id } lookup;
x in ^keys membership over a lifted set.
## select — projection
select a, alias: expr, nested: _nodes collect { … }
bare key → your property (scalar-authored → scalar; list → array); _name →
the engine's field (_id, _path, _title, …); _data.<k> is the bare key's read;
a call/arithmetic needs an alias (n: size(tags)) unless in values mode;
an output key of _id/_path other than the attribute itself needs an alias.
_raw whole file bytes (docs) — docs_read is cheaper for one doc.
distinct select distinct _type (dedup hits by projected value)
values select <ONE item> values → bare values instead of records:
select distinct type values from docs → values: ["hub","lab-note",…]
select _path values from docs → values: ["/index.md", …]
tags: tags collect { $it values } → a plain array
h: _nodes first { _name values where _kind == "md:section" order by first_ordinal }
entries(x) a record as a collection ($key / $it per entry), key order:
select fm: entries(_data) collect { k: $key, v: $it }
where entries(_frontmatter) exists { where $key == "era" and $it > 1600 }
entries(_inline), entries(_attrs), a nested map, or a list (index keys)
$semantic_score is not a field — project semantic("…") under an alias instead.
## flatten (OQX 0.25)
flatten <expr> [{ … }] [as <name>] after follow, before group by; may repeat
Splits one value into rows. Every hit is still the PARENT row — its _id/_path
repeat per element, as Dataview's TABLE … FLATTEN — with <name> bound beside
its fields at the same depth: an array → one row per element (an empty list
yields none), a missing field → one row with the name absent (never dropped),
an object or a scalar → one row with the value. A plain navigation names
itself (flatten tags rebinds tags to each element); anything else needs as.
The block is the ordinary receiver block over the elements (where, a
projection, order by, limit, distinct), applied first. Later clauses read the
name — a later flatten (flatten jobs as j flatten j.tags as t), group by,
having, select, order by; where runs before it. A flattened query pages by
position (cursor null); as _id / as _path is refused (the envelope reads them).
select _path, tags from docs where type == "process" flatten tags order by _path, tags
select _path, t: b._text from docs flatten _blocks { where _type == "task" and not checked } as b
select tags, n: count() from docs flatten tags group by tags order by count() desc
## order by, limit/offset, pagination
order by era desc, _path asc absent values sort LAST (asc); ties → (_path, _id)
limit N / offset N define the result SET (integer literals);
the tool's limit/cursor then page WITHIN it
cursor opaque keyset on (_path, _id); a custom order by
disables it (you get the top page + truncated)
## Graph traversal (follow) and links
The citation graph is the LINK ROWS walked through a destination block:
_dst is a link's destination document (absent for an external or dangling
link — it then contributes no successor), _doc its source.
from docs where <seed> follow _out_links collect { from _dst } docs the seed links TO
from docs where <seed> follow _in_links collect { from _doc } backlinks
follow _out_links collect { from _dst depth 3 } bound (1..8, default 8) — a clause of the block
follow _out_links collect { from _dst where layer == "canon" } keep only matching successors
follow _in_links collect { from _doc where ^_path in list(before) } ^ = the row being expanded
(^^ = the walk's enclosing scope)
follow _out_links collect { from _dst where _predicate == "depends_on" } one predicate only
follow _out_links collect { from _dst }, _in_links collect { from _doc }
several destinations: their union
(one step dedups by identity)
follow ^docs { where _out_links exists { where _dst_path == ^^_path } }
a destination block over the root's
documents, re-read per frontier row
(^ = that row) = the backlinks walk
follow _out_links collect { from _dst frontier type == "practitioner" } cut, keeping the frontier row
from docs where type == "process" frontier !verified follow _out_links collect { from _dst }
the BODY's frontier applies to the seeds:
an unverified seed is kept, not expanded
follow distinct _out_links collect { from _dst } / … { $key: group from _dst } identity control ($key)
follow refs(before), ^docs { where ^_path in list(after) } forward over `before`, backward
over every doc whose `after` names me
Per-occurrence metadata: $depth (seed = 1), $stop (interior|leaf|frontier|
depth|cycle), $leaf/$frontier, $ordinal (a deterministic 1..N rank over the
walk, ordered by depth then path — `order by $ordinal` for walk order,
`where $ordinal <= N` for a deterministic budget cut), $key — usable in
select/order by and the top-level post-walk where (a where conjunct that
reads one filters the walk's RESULT; the other conjuncts remain the seed
predicate), NOT metadata of the candidates a destination block's where reads,
nor valid inside a consumer directive. Cyclic graphs are safe: a revisited
node is admitted once as a `$stop == "cycle"` occurrence and never
re-expanded. The default keeps one occurrence per distinct walk path;
`follow distinct` dedups reached rows by identity.
Destination blocks (OQX 0.20): a brace after ANY destination opens that
destination's block — _children { depth 2 }, refs(x) { depth 2 }, _out_links
collect { from _dst depth 2 }, ^docs { where ^_path in list(after) } — with the
0.17 body rules (where-first, projections) and two clauses of its own after
`where`: frontier <pred> (occurrences reached through this destination that
match are frontiers; ^ there is the walk's enclosing scope) and depth <n>
(this destination expands a frontier row only while $depth < n). The body
that carries the follow may write frontier <pred> too, after its where and
before follow: it applies to the SEEDS (read like the body's where) — a
matching seed stays in the result with $stop == "frontier" and is not
expanded; each destination's frontier is its own. There is no options block;
the old `by` is $key. A bare follow ^docs (no brace, no consumer) is an
error naming both block spellings.
Type-preserving relations: a block's _children; a section node's _children and
_subsections; the citation graph is _out_links collect { from _dst } / _in_links collect { from _doc }.
from links where _dst_path == "/notes/x.md" inbound links to a doc, as rows
select d: _dst._title, k: _dst_kind from links the destination document's title
from docs where _out_links exists { where _dst.layer == "draft" } links into drafts
select b: _in_links collect distinct { from _doc } from docs backlinks as { _id, _path }
select _doc_id, _dst_path from links where _predicate == "depends_on"
from docs where not _in_links exists { } orphans (nothing links in)
(The `graph` tool is a convenience wrapper that compiles to a follow query.)
## semantic grain
from docs ranks whole DOCUMENTS (one hit per file); from blocks ranks PASSAGES.
text()/where prune the candidate set; only order by reweights.
## Examples
from docs where layer == "working"
from docs where _path.startsWith("/guides/") and _updated_at >= "2026-08-01"
select layer, tags from docs where "pricing" in list(tags)
from docs where _inline.owner == "alice"
from docs where _title.lower().contains("q3 plan")
from docs where _format == "yaml"
from docs where type == "practitioner" order by era desc limit 2
from docs where _nodes none { where _kind == "md:task" and not checked }
from docs where tags exists { where $it == "tria-prima" }
select fm: entries(_data) collect { k: $key, v: $it } from docs where _path == "/x.md"
select owner_id, owner: ^nodes single { where _kind == "person" and _attrs.id == ^owner_id } from docs
from blocks where _type == "task" and not checked and under_heading("Launch") and _doc.layer == "working"
from blocks where _type == "paragraph" and has_edge("references", "d_92aaaaa")
select _path, $depth from docs where _path == "/timeline/review.md" follow refs(before), refs(after)
select _path, next: ^docs collect { _path where ^_path in list(before) } from docs where type == "milestone"
select _path, n: size(^docs), repo: ^_id from docs limit 1
select path, title from docs where has(path)
docs { type == "substance" and not verified }
select subject, process: ^docs[slug == ^subject]!._path from docs where type == "lab-note"
select _path, back: _nodes first { select p: 0^docs collect { _path where ^^_path in list(after) } values } from docs where type == "milestone"
select _path, prior: refs(before) collect { _path, when } from docs where type == "milestone"
from blocks order by semantic("identity preservation across edits") desc limit 10
select _uri from links where _dst_kind == "external"
docs count { where layer == "canon" }
select _seq, _origin, _ts, _hash from revisions where _path == "/processes/calcination.md" order by _seq desc
from docs where _revisions count { where _origin == "api" } > 0
select p: _path, d: _doc._path from revisions where _doc == null
select layer, n: count(), titles: collect(_title), open: rows count { where _nodes exists { where _kind == "md:task" and not checked } } from docs group by layer having count() > 1 order by count() desc
select _path, tag: tags from docs where "process" in tags flatten tags order by _path, tags
Common mistake: `$path`/`$id` — omgbase has no $ names; the engine's location and
identity are `_path`/`_id`. Another: the 4.x bare `path`/`id`/`title`/`tags` — a
bare `path` is now YOUR `path:` key (absent when you wrote none, with a hint
pointing at `_path`); the engine's are `_path`/`_id`/`_title`/`_tags`; `select
data.path` is gone — write `path`. Another: `where tags == "x"` on a list-valued
key is false — write `"x" in tags`. Another: `select _path: _doc._path` — alias it
(`p: _doc._path`).
"##;
pub const QUERY_SYNTAX_SECTIONS: &[(&str, &str)] = &[
("sugar", "Sugar (OQX 0.17)"),
("paths", "Paths"),
("targets", "Targets & field namespaces"),
(
"scoping",
"Scoping (ADR-015; the root row since surface 2.0): bare names are LOCAL",
),
("scalars", "Scalar semantics (@omgbase/oqx)"),
("predicates", "Domain predicates (omgbase)"),
("nested", "Nested queries & consumers"),
("select", "select — projection"),
("flatten", "flatten (OQX 0.25)"),
("order", "order by, limit/offset, pagination"),
("follow", "Graph traversal (follow) and links"),
("semantic", "semantic grain"),
("examples", "Examples"),
];
pub const QUERY_SYNTAX_SECTION_NAMES: &[&str] = &[
"sugar",
"paths",
"targets",
"scoping",
"scalars",
"predicates",
"nested",
"select",
"flatten",
"order",
"follow",
"semantic",
"examples",
"all",
];
#[must_use]
pub fn query_syntax_section(slug: &str) -> Option<String> {
if slug == "all" {
return Some(QUERY_SYNTAX.to_owned());
}
let (_, heading) = QUERY_SYNTAX_SECTIONS.iter().find(|(s, _)| *s == slug)?;
let lines: Vec<&str> = QUERY_SYNTAX.split('\n').collect();
let title = format!("## {heading}");
let start = lines.iter().position(|l| *l == title)?;
let mut end = start + 1;
while end < lines.len() && !lines[end].starts_with("## ") {
end += 1;
}
while end > start + 1 && lines[end - 1].trim().is_empty() {
end -= 1;
}
let mut text = lines[start..end].join("\n");
text.push('\n');
Some(text)
}
pub const QUERY_SYNTAX_CARD: &str = r#"# query — OQX card. Chapters: query_syntax { section: "<name>" }; the manual: { section: "all" }
One OQX string; limit/cursor page the hits; bare: true drops the { _id, _path } envelope.
[select <items>] from <target> [where <pred>] [follow <dest> { … }] [order by <expr> [desc]] [limit N] [offset N]
<target> count|exists|none|first|single { <block> } a scalar or one row
CLAUSE ORDER IS FIXED: select, from, where, frontier, follow, flatten, group by, having, order by, limit, offset.
and / or / not. Comparisons strict and typed; null == a missing field. A list key: "x" in tags, never tags == "x".
Strings .contains("s") .startsWith("p") .lower(); x ~* "re" (ci regex), x !~ "re". Dates date("today"),
date(due) < date("today").minus(dur("30 days")), _updated_at >= "2026-08-01".
NAMES: a BARE name is YOURS (frontmatter, inline key:: value, a block's attrs); an UNDERSCORE name is the ENGINE's;
$ is OQX's ($it $key $depth $stop $ordinal). title tags path id are your keys; _title _tags _path _id the engine's.
docs _id _path _raw _hash _updated_at _created_at _title _tags _format | _data _frontmatter _inline | _blocks _nodes _out_links _in_links _revisions
blocks _id _path _doc_id _raw _hash _type _text _attrs _depth _ordinal | _doc | _children _nodes _out_links _section
nodes _id _path _doc_id _block_id _kind _name _value _attrs | _doc _block | _blocks _subsections _children
links _id _path _doc_id _block_id _via_id _dst_id _dst_path _uri _dst_kind _predicate _provenance _anchor _field | _doc _dst _block
revisions _id _path _rev _seq _doc_id _commit _ts _origin _actor _reason _hash _is_current _archived | _doc
Handles _data.<k> ≡ <k>, _frontmatter.<k> / _inline.<k>; entries(_data) the bag as rows ($key, $it). Reach-through
_doc.<k> the owning document, _dst.<k> a link's target. Scoping: a bare name is the CURRENT row; ^name one scope
out; ^docs ^blocks ^nodes ^links the repository from a row. An unknown bare name is absent plus a hints line.
NESTED: <receiver> collect|exists|none|count|first|single { [select …] [where …] [order by …] [limit N] } over a
relation, a list key, entries(x) or ^docs; no consumer = collect, a leading predicate = where: docs { type == "x" }.
FOLLOW: from docs where <seed> follow _out_links collect { from _dst depth 3 } links out; _in_links { from _doc }
backlinks; refs(before) a property's documents; $depth per row. SEARCH: text("words") prunes; order by semantic("…") desc.
from docs where layer == "working"
select _title, stage from docs where "process" in tags order by _path limit 5
select _type, _text from blocks where _path == "/processes/calcination.md"
select _path, _value from nodes where _kind == "md:task" and not checked
select _path, _block_id from links where _dst_path == "/processes/calcination.md"
from docs where _title.lower().contains("salt") and _updated_at >= "2026-01-01"
from docs where _nodes exists { where _kind == "md:task" and not checked }
select _path, $depth from docs where _path == "/timeline/review.md" follow refs(before), refs(after)
docs count { where layer == "canon" }
select _seq, _ts, _hash from revisions where _path == "/processes/calcination.md" order by _seq desc
select tags, n: count() from docs flatten tags group by tags order by count() desc
Every error is filter_invalid naming what to change; when the engine can correct the call, data.fix IS the call.
Chapters: sugar paths targets scoping scalars predicates nested select flatten order follow semantic examples; all.
"#;
#[cfg(test)]
mod tests {
use super::{
QUERY_SYNTAX, QUERY_SYNTAX_CARD, QUERY_SYNTAX_SECTION_NAMES, QUERY_SYNTAX_SECTIONS,
query_syntax_section,
};
const TARGETS: [&str; 4] = ["docs", "blocks", "nodes", "links"];
const CONSUMERS: [&str; 7] = [
"collect", "exists", "none", "count", "first", "single", "distinct",
];
fn has_top_level_from(q: &str) -> bool {
let chars: Vec<char> = q.chars().collect();
let (mut depth, mut quote): (i32, Option<char>) = (0, None);
for (i, &c) in chars.iter().enumerate() {
if let Some(open) = quote {
if c == open {
quote = None;
}
continue;
}
match c {
'"' | '\'' => quote = Some(c),
'{' => depth += 1,
'}' => depth -= 1,
_ if depth == 0 && chars[i..].starts_with(&[' ', 'f', 'r', 'o', 'm', ' ']) => {
return true;
}
_ => {}
}
}
false
}
fn has_placeholder(q: &str) -> bool {
let chars: Vec<char> = q.chars().collect();
for (i, &c) in chars.iter().enumerate() {
if c != '<' || !chars.get(i + 1).is_some_and(|n| n.is_ascii_alphabetic()) {
continue;
}
let mut j = i + 1;
while j < chars.len()
&& (chars[j].is_ascii_alphabetic() || matches!(chars[j], '_' | '|' | ' ' | '…'))
{
j += 1;
}
if chars.get(j) == Some(&'>') {
return true;
}
}
false
}
fn is_prose(q: &str) -> bool {
q.contains(". ")
|| q.contains("; ")
|| q.ends_with('.')
|| q.ends_with(':')
|| q.ends_with(';')
}
fn root_consumer(q: &str) -> bool {
let mut it = q.splitn(3, ' ');
let (Some(target), Some(next)) = (it.next(), it.next()) else {
return false;
};
TARGETS.contains(&target) && (CONSUMERS.contains(&next) || next.starts_with('{'))
}
fn extract(text: &str) -> (Vec<(usize, String, String)>, Vec<String>) {
let (mut queries, mut fragments) = (Vec::new(), Vec::new());
let mut section = String::new();
for (i, raw) in text.lines().enumerate() {
if let Some(h) = raw.strip_prefix("## ") {
section = h.trim().to_owned();
continue;
}
if !raw.starts_with(" ") || raw.trim().is_empty() {
continue;
}
let trimmed = raw.trim();
if section == "Examples" {
queries.push((i + 1, section.clone(), trimmed.to_owned()));
continue;
}
let query = trimmed.split(" ").next().unwrap_or("").trim();
if has_placeholder(query) || query.contains('…') || is_prose(query) {
continue;
}
if !(query.starts_with("select ") || query.starts_with("from ") || root_consumer(query))
{
continue;
}
if query.starts_with("select ") && !has_top_level_from(query) {
fragments.push(query.to_owned());
continue;
}
queries.push((i + 1, section.clone(), query.to_owned()));
}
(queries, fragments)
}
#[test]
fn extraction_is_substantial() {
let (queries, fragments) = extract(QUERY_SYNTAX);
let in_examples = queries.iter().filter(|q| q.1 == "Examples").count();
assert!(in_examples >= 20, "{in_examples} examples");
assert!(
queries.len() - in_examples >= 10,
"{} outside Examples",
queries.len() - in_examples
);
for f in &fragments {
assert!(
!TARGETS.iter().any(|t| f.contains(&format!(" from {t}"))),
"a fragment must not carry a top-level from: {f}"
);
}
}
#[test]
fn every_example_parses() {
let (queries, _) = extract(QUERY_SYNTAX);
let failures: Vec<String> = queries
.iter()
.filter_map(|(line, section, q)| {
oqx::parse_string(q)
.err()
.map(|e| format!("line {line} ({section}): {q}\n {e}"))
})
.collect();
assert!(
failures.is_empty(),
"query_syntax examples that do not parse:\n{}",
failures.join("\n")
);
}
#[test]
fn error_examples_live_in_prose() {
let (queries, _) = extract(QUERY_SYNTAX);
for bad in [
"era > 1600 from docs",
"from docs count",
"from docs where \"pricing\" in list(tags) select layer, tags",
] {
assert!(oqx::parse_string(bad).is_err());
assert!(
!queries.iter().any(|q| q.2 == bad),
"{bad} is an error example and must stay in prose"
);
}
}
#[test]
fn clause_order_sentence_matches_the_parser() {
let e = oqx::parse_string("from docs where x select y")
.unwrap_err()
.to_string();
let (_, rest) = e
.split_once("OQX clause order is ")
.expect("the parser names the order");
let order = rest
.split(" (")
.next()
.unwrap()
.trim_end_matches(['.', ' ']);
assert!(
QUERY_SYNTAX.contains(&format!("CLAUSE ORDER IS FIXED: {order}")),
"the text's order sentence drifted from the parser's ({order})"
);
}
fn reference_ts() -> Option<String> {
let path = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../packages/core/src/mcp/reference.ts"
);
std::fs::read_to_string(path).ok()
}
fn ts_literal(ts: &str, name: &str) -> String {
let open = format!("export const {name} = `");
let start = ts.find(&open).expect("the TS export") + open.len();
let end = start + ts[start..].find("\n`;").expect("the TS literal's end") + 1;
let mut text = String::new();
let mut chars = ts[start..end].chars();
while let Some(c) = chars.next() {
if c == '\\' {
if let Some(n) = chars.next() {
text.push(n);
}
} else {
text.push(c);
}
}
text
}
#[test]
fn twin_is_byte_identical_to_the_reference() {
let Some(ts) = reference_ts() else {
return;
};
assert!(
ts_literal(&ts, "QUERY_SYNTAX") == QUERY_SYNTAX,
"crates/omgbase-surface/src/reference.rs drifted from packages/core/src/mcp/reference.ts"
);
}
#[test]
fn card_and_sections_are_byte_identical_to_the_reference() {
let Some(ts) = reference_ts() else {
return;
};
assert!(
ts_literal(&ts, "QUERY_SYNTAX_CARD") == QUERY_SYNTAX_CARD,
"QUERY_SYNTAX_CARD drifted from packages/core/src/mcp/reference.ts"
);
let table_at = ts
.find("export const QUERY_SYNTAX_SECTIONS")
.expect("the TS section table");
let mut cursor = table_at;
for (slug, heading) in QUERY_SYNTAX_SECTIONS {
let pair = format!("{{ slug: \"{slug}\", heading: \"{heading}\" }}");
let at = ts[cursor..]
.find(&pair)
.unwrap_or_else(|| panic!("the TS table lacks {pair} after the previous pair"));
cursor += at + pair.len();
}
let pairs = ts[table_at..cursor].matches("{ slug: \"").count();
assert_eq!(
pairs,
QUERY_SYNTAX_SECTIONS.len(),
"the TS table has pairs this twin lacks"
);
assert!(
!ts[cursor..ts[cursor..].find("];").unwrap() + cursor].contains("{ slug: \""),
"the TS table has pairs past this twin's last"
);
}
#[test]
fn sections_are_the_headings_in_order_and_partition_the_manual() {
let headings: Vec<&str> = QUERY_SYNTAX
.lines()
.filter_map(|l| l.strip_prefix("## "))
.collect();
let table: Vec<&str> = QUERY_SYNTAX_SECTIONS.iter().map(|(_, h)| *h).collect();
assert_eq!(table, headings);
let mut names: Vec<&str> = QUERY_SYNTAX_SECTIONS.iter().map(|(s, _)| *s).collect();
names.push("all");
assert_eq!(names, QUERY_SYNTAX_SECTION_NAMES);
let mut joined = Vec::new();
for (slug, heading) in QUERY_SYNTAX_SECTIONS {
assert!(slug.chars().all(|c| c.is_ascii_lowercase()), "{slug}");
let text = query_syntax_section(slug).expect(slug);
assert!(text.starts_with(&format!("## {heading}\n")), "{slug}");
assert!(text.ends_with('\n') && !text.ends_with("\n\n"), "{slug}");
assert!(!text.contains("\n## "), "{slug} holds a second heading");
joined.push(text);
}
let body = &QUERY_SYNTAX[QUERY_SYNTAX.find("\n## ").unwrap() + 1..];
let squash = |s: &str| {
let mut out = String::new();
for l in s.split('\n').filter(|l| !l.is_empty()) {
out.push_str(l);
out.push('\n');
}
out
};
assert_eq!(squash(&joined.join("\n")), squash(body));
assert_eq!(query_syntax_section("all").as_deref(), Some(QUERY_SYNTAX));
assert_eq!(query_syntax_section("nope"), None);
assert!(QUERY_SYNTAX_CARD.contains("CLAUSE ORDER IS FIXED: select, from, where"));
for (slug, _) in QUERY_SYNTAX_SECTIONS {
assert!(QUERY_SYNTAX_CARD.contains(slug), "the card names {slug}");
}
}
#[test]
fn every_card_example_parses() {
let (queries, _) = extract(QUERY_SYNTAX_CARD);
assert!(queries.len() >= 10, "{} card examples", queries.len());
let failures: Vec<String> = queries
.iter()
.filter_map(|(line, _, q)| {
oqx::parse_string(q)
.err()
.map(|e| format!("line {line}: {q}\n {e}"))
})
.collect();
assert!(
failures.is_empty(),
"card examples that do not parse:\n{}",
failures.join("\n")
);
}
}