Expand description
The treebank vocabulary (notes/DESIGN.md §3), as code.
Two tiers with different physics, dictated by what tree-sitter’s supertype mechanism can express:
- Table tier — real supertype rules threaded through a grammar’s productions. Occurrence-level semantics, enforced at generate time, natively queryable.
- Facet tier — roles that cross-cut derivations and therefore cannot
be supertypes. Shipped as a
roles.jsonmanifest per grammar crate (type-level membership) and expanded into concrete alternations at query-load time byexpand.
A term’s tier is a property of the grammar, not of the vocabulary:
terms listed in either_tier may be delivered by either mechanism, and
each grammar picks. The table tier is stronger and is the default; a
grammar demotes a term to a facet only when its language partitions the
position (Python orders parameters, so _parameter cannot be one
alternation without the grammar accepting def f(a=1, b)). Demotion is
sound precisely when every member of the term is a concrete node type
that occurs nowhere else, since then type-level and occurrence-level
membership select the same nodes; see notes/DESIGN.md §3.4.
The vocabulary itself lives in vocabulary/vocabulary.json, embedded
here and re-exported to JavaScript by vocabulary/supertypes.js, so the
grammars and this crate can never disagree about what the vocabulary is.
Re-exports§
pub use pack::Pack;
Modules§
- check
- The vocabulary-conformance checker (
treebank roles, notes/DESIGN.md §3.3). - expand
- Facet query expansion. Facet roles are not grammar rules, so a query
like
(_callable name: (_name) @n)cannot run against the parser as written; this rewrites every facet pattern into the concrete alternation the manifest defines: - fetch
- Fetch a grammar, so using one is a line of code rather than a build step.
- node_
types - A minimal reading of tree-sitter’s generated
src/node-types.json: which named node types exist, and which of them are supertypes (the entries carrying asubtypeslist). - pack
- Load a Treebank wasm pack and parse with it.
- roles
- The
roles.jsonfacet manifest each grammar crate ships (notes/DESIGN.md §3.1). Facets are type-level: a node type is_callablewherever it occurs. The manifest also carries the grammar’suncategorisedlist — every named node outside the vocabulary, each with a reason — so that nothing is silently outside it (§3.3 rule 2).
Structs§
- Term
- One vocabulary term: a name (always underscore-prefixed) and its one-line definition.
- Vocabulary
- The closed vocabulary. A grammar may omit terms its language lacks; it may not invent terms.
Functions§
- vocabulary
- The vocabulary this build of treebank carries. Parsing the embedded
JSON cannot fail for a released crate; the unit tests parse it too, so a
malformed edit fails
cargo testbefore it can fail a consumer.