Struct Grammar 

#[non_exhaustive]
pub struct Grammar {

    pub name: String,
    pub start_symbol: String,
    pub rules: BTreeMap<String, Production>,
    pub supertypes: HashSet<String>,
    pub extras: HashSet<String>,
    pub inline_rules: HashSet<String>,
    pub subtypes: HashMap<String, HashSet<String>>,
    pub yield_sets: HashMap<String, HashSet<String>>,
    pub node_type_children: HashMap<String, HashSet<String>>,
    pub node_type_field_children: HashMap<String, HashMap<String, HashSet<String>>>,
    pub node_type_nonfield_children: HashMap<String, HashSet<String>>,
    pub external_alias_map: HashMap<String, String>,
    pub token_roles: HashMap<String, HashMap<String, TokenRole>>,
    pub indent_triggers: HashSet<(String, String)>,
    pub line_comment_prefixes: Vec<String>,
    pub trailing_break_markers: Vec<String>,
    pub trailing_break_on_whitespace: bool,
    pub top_level_text_admits_newline: bool,
    pub external_indent_opens: HashSet<String>,
    pub external_indent_closes: HashSet<String>,
    pub external_newlines: HashSet<String>,
    pub external_semicolons: HashSet<String>,
    pub external_bracket_opens: HashSet<String>,
    pub external_bracket_closes: HashSet<String>,
    pub external_content_kinds: HashSet<String>,
    pub string_content_kinds: HashSet<String>,
    pub synthetic_indent_rules: HashSet<String>,
    pub named_alias_map: HashMap<String, String>,
    pub named_alias_sources: HashMap<String, Vec<String>>,
    pub leading_space_terminals: HashSet<String>,
    pub line_rest_kinds: HashSet<String>,
    pub immediate_token_alias_kinds: HashSet<String>,
    pub external_close_text: HashMap<String, String>,
    /* private fields */
}

A grammar’s production-rule table, deserialized from grammar.json.

Only the fields the emitter consumes are decoded; precedences, conflicts, externals, and other parser-only metadata are ignored.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

name: String

Grammar name (e.g. "rust", "typescript").

start_symbol: String

The grammar’s start symbol: the first rule as written in grammar.json (tree-sitter’s entry point). Recovered from the raw bytes because rules is a BTreeMap that loses the original insertion order.

rules: BTreeMap<String, Production>

Map from rule name (a vertex kind on the schema side) to production. Entries are kept in lexical order so iteration is deterministic.

supertypes: HashSet<String>

Supertypes declared in the grammar’s supertypes field. A supertype is a rule whose body is a CHOICE of SYMBOL references; tree-sitter parsers report a node’s kind as one of the subtypes (e.g. identifier, typed_parameter) rather than the supertype name (parameter), so the emitter needs to know that a child kind in a subtype set should match the supertype name when a SYMBOL references it.

extras: HashSet<String>

Tree-sitter extras rules: the named symbols (typically comments) that tree-sitter skips at parse time but records as children of the surrounding vertex. They appear nowhere in the production grammar, so the rule walker cannot reconcile them against the cursor — the emit pass therefore drains them as a side channel: at vertex entry and between REPEAT iterations any leading extras-kind edges are consumed and emitted directly. The set is populated at Grammar::from_bytes by collecting every SYMBOL { name } and named ALIAS { value, named: true } under the top-level extras array. Pattern-only extras (e.g. \s whitespace) are not vertex kinds and are excluded.

inline_rules: HashSet<String>

Tree-sitter inline rules: named rules the generator splices into every referencing production rather than emitting as their own node. An inlined rule’s children (its FIELDs and bare SYMBOL members) are promoted to be direct children of the referencing vertex, so on the schema side there is no child vertex of the inlined rule’s kind. When the emit walk hits a SYMBOL member naming an inlined rule it must therefore expand that rule’s body inline against the current cursor (the same treatment a hidden _-prefixed rule gets), or the inlined members’ edges are dropped (brightscript sub_impl/function_impl drop parameters/body/ end_statement). Populated from grammar.json’s top-level inline array.

subtypes: HashMap<String, HashSet<String>>

Precomputed subtyping closure: subtypes[symbol_name] is the set of vertex kinds that satisfy a SYMBOL symbol_name reference on the schema side.

Built once at Grammar::from_bytes time by walking each hidden rule (_-prefixed), declared supertype, and named ALIAS { value: K, ... } production to its leaf SYMBOLs and recording the closure. This replaces the prior heuristic kind_satisfies_symbol that walked the rule body on every query: lookups are now O(1) and the relation is exactly the transitive closure of “is reachable via hidden / supertype / alias dispatch”, with no over-expansion through non-hidden non-supertype rule references.

yield_sets: HashMap<String, HashSet<String>>

Precomputed Yield sets: yield_sets[rule_name] is the set of concrete vertex kinds that can appear as the first named child when that rule’s production is taken.

Defined inductively:

• Yield(SYMBOL S) where S is hidden/supertype = Yield(rules[S])
• Yield(SYMBOL S) where S is concrete = {S}
• Yield(SEQ [M1, ...]) = Yield(M1) (only first member)
• Yield(CHOICE [M1, ..., Mn]) = ⋃ Yield(Mi)
• Yield(OPTIONAL { c }) = Yield(c) ∪ {ε}
• Yield(BLANK) = {ε}
• Wrappers (PREC*, TOKEN, FIELD, REPEAT, etc.) = Yield(content)
• Yield(STRING) = Yield(PATTERN) = ∅
• Yield(ALIAS { value: V, named: true }) = {V}

Epsilon is represented as the empty string "".

node_type_children: HashMap<String, HashSet<String>>

Child kinds allowed per parent kind, derived from node-types.json. Maps parent kind to the set of ALL named child kinds that tree-sitter’s parser can produce for that parent (from both children.types and fields.*.types). Used by augment_subtypes_from_node_types to close the grammar/parser divergence gap.

node_type_field_children: HashMap<String, HashMap<String, HashSet<String>>>

Per-field child kinds from node-types.json: maps parent kind → field name → set of child kinds. Used by the augmentation to restrict subtype edges to structurally matching positions.

node_type_nonfield_children: HashMap<String, HashSet<String>>

Non-field child kinds from node-types.json: maps parent kind → set of child kinds that appear in children.types (not in any field).

external_alias_map: HashMap<String, String>

Anonymous ALIAS values for external scanner tokens. Maps external symbol name (e.g. _ternary_qmark) to the ALIAS value string (e.g. "?"). Built by scanning grammar.json rule bodies for ALIAS { content: SYMBOL S, named: false, value: V } where S has no grammar rule.

token_roles: HashMap<String, HashMap<String, TokenRole>>

Per-rule token role classification. Maps rule name to a map of STRING value to its structural role in that rule. Derived at construction time by analyzing each rule’s SEQ structure to identify bracket pairs, separators, keywords, and operators.

indent_triggers: HashSet<(String, String)>

Set of (rule_name, open_bracket_value) pairs where the bracket triggers indentation (the content between open and close contains REPEAT/REPEAT1). Block-level constructs like statement_block use indenting brackets; inline constructs like interpolation do not.

line_comment_prefixes: Vec<String>

Line-comment prefixes extracted from the grammar’s extras. Each prefix is a STRING value from a TOKEN(SEQ [STRING prefix, PATTERN ...]) pattern in the extras array, verified to be an extras rule. Used by the layout pass to insert a newline after comment Lit tokens.

trailing_break_markers: Vec<String>

Bare literal markers that, when emitted as the final token of the output, must NOT be followed by the customary end-of-output newline.

Derived from productions of the shape SEQ[CHOICE[.. bare lit ..], <newline-leading>] — tree-sitter’s “hard line break” idiom (markdown_inline’s hard_line_break = SEQ[CHOICE["\\" | _whitespace_ge_2], _soft_line_break]). A trailing backslash (or trailing whitespace, see trailing_break_on_whitespace) is plain content on its own; only a following newline turns it into a line-break node. The end-of-output newline the layout fold appends would therefore manufacture a phantom break node on re-parse, so it is suppressed when the output ends with one of these markers.

Restricted to SINGLE-character non-alphanumeric literals so the rule fires only on genuine standalone break markers (\), never on keyword/identifier-led line constructs (posting, declaration, go_directive) whose leading literal is substantive content.

trailing_break_on_whitespace: bool

Whether the grammar has a hard-line-break production whose leading alternative is a whitespace-only PATTERN (markdown_inline’s _whitespace_ge_2). When set, a final emitted token ending in trailing spaces/tabs also suppresses the end-of-output newline.

top_level_text_admits_newline: bool

Whether the grammar’s top-level document repeat directly admits a free-text content node whose pattern matches a bare newline (template / markup grammars: liquid’s template_content = REPEAT1([^{]+ | ...), twig’s content, eex’s text). For such grammars a lone trailing newline appended at end of output is captured as an extra content node on re-parse, inflating the kind-multiset, so the end-of-output newline is suppressed.

Derived narrowly: the content rule must be a DIRECT child of the start symbol’s top-level REPEAT (through hidden symbols / CHOICE), so the rule fires only on genuine document text, never on the newline-admitting negated classes inside comments or string fragments (which are nested under delimiters, not document nodes).

external_indent_opens: HashSet<String>

External tokens that produce indent-open layout actions. Identified by tree-sitter naming convention: names ending with _indent or equal to _indent.

external_indent_closes: HashSet<String>

External tokens that produce indent-close layout actions.

external_newlines: HashSet<String>

External tokens that produce line breaks.

external_semicolons: HashSet<String>

External tokens equivalent to semicolons.

external_bracket_opens: HashSet<String>

External scanner tokens that open a delimiter pair around content (e.g. string_start in SEQ[string_start, REPEAT(content), string_end]). Derived structurally; emitted tight on the inside ('hello', not ' hello ').

external_bracket_closes: HashSet<String>

External scanner tokens that close a delimiter pair around content (e.g. string_end). Emitted tight on the inside.

external_content_kinds: HashSet<String>

Visible (non-_-prefixed) external scanner tokens that are the captured content between a pair of string/heredoc delimiters in a SEQ[open_ext, REPEAT(content..), close_ext] rule (ruby string_content / heredoc_content, regex content, command-string content). Such a token’s text IS the literal source bytes between the delimiters: the layout pass must NOT insert a sibling-separation space around it ("bar", not " bar "), or a space folds into the captured text on re-parse and accretes one space per emit. Derived structurally from the same delimiter shape as external_bracket_opens.

string_content_kinds: HashSet<String>

Named content kinds that sit between a matched pair of quote delimiters spelled as literal STRING tokens, in a rule shaped SEQ[STRING q, REPEAT(CHOICE[content..]), STRING q] (the same quote opens and closes). The CSS string_value and the C# / Java string_literal are the canonical cases: the body is a REPEAT over CHOICE[string_content (an ALIAS over a PATTERN), escape_sequence]. Each such content / escape leaf carries the verbatim source bytes and must emit tight on both sides ("ab\t", not "ab \t "), exactly like external_content_kinds but for the STRING-delimited (rather than external-delimited) string shape that classify_external_bracket_delimiters skips (it only matches external delimiters). Derived purely from grammar structure (the matched-literal-quote envelope), so it stays in the generic emitter.

synthetic_indent_rules: HashSet<String>

Rule names that are indented blocks whose opening _indent lives in a (hidden) parent rule rather than the rule itself: their body references an external indent-close token (_dedent) but no indent-open token. The parser reaches such a block vertex directly (the hidden _suite wrapper carrying the _indent is not a vertex), so the emitter must synthesize the opening indent (def f(): then an indented body) when it walks the rule.

named_alias_map: HashMap<String, String>

Named alias map: maps alias value to source symbol name. When a vertex kind has no direct grammar rule, this map resolves ALIAS { content: SYMBOL source, named: true, value: alias } so the emitter can walk the source rule with proper token roles.

named_alias_sources: HashMap<String, Vec<String>>

Every source rule that aliases to a given kind, in grammar order. A kind can be the value of several distinct ALIAS sites (cpp function_definition is the alias value of inline_method_definition, constructor_or_destructor_definition, operator_cast_definition, …, AND has its own function_definition rule). When the vertex’s own rule cannot consume one of its child edges (a parser.c/grammar.json desync where the collapsed-kind rule omits constructor-only members like field_initializer_list), the emitter falls back to the alias source whose production does admit the child set.

leading_space_terminals: HashSet<String>

Named terminal kinds whose underlying PATTERN can match a leading space (e.g. INI’s setting_value = PATTERN ".+"). A layout space emitted before such a terminal would fold into its captured text on re-parse and accrete one space per emit, so the emitter hugs them to their predecessor. See pattern_absorbs_leading_space.

line_rest_kinds: HashSet<String>

Named terminal kinds whose underlying PATTERN runs to the end of the source line (an unbounded trailing .* / .+, e.g. JS’s hash_bang_line = #!.*). Like a line comment, such a token absorbs any text that follows it on the same line, so the layout pass emits a newline after it: otherwise the next sibling re-parses as part of the token. See is_rest_of_line_pattern.

immediate_token_alias_kinds: HashSet<String>

Named alias values whose ALIAS content reduces to an IMMEDIATE_TOKEN (e.g. C’s char_literal body ALIAS{IMMEDIATE_TOKEN PATTERN "[^\n']", value: "character"}). The lexer admits such a token only with no preceding whitespace, so the emitter hugs it to its predecessor: the alias-value carries no grammar rule, so the rule-head IMMEDIATE_TOKEN no-space check in emit_vertex never fires for it. Emitting these leaves tight keeps 'hey' from re-spacing to ' h e y' (whose spaces re-parse as extra character nodes).

external_close_text: HashMap<String, String>

Text to emit for an external closing delimiter whose matching opener is a literal STRING (a rule shaped SEQ[STRING q, body.., EXTERNAL close], the asymmetric twin of the all-external and all-STRING delimiter shapes). TOML’s _multiline_basic_string = SEQ[STRING """, REPEAT(..), _multiline_basic_string_end] is the canonical case: the open """ is a grammar literal, but the close is a scanner external with no rule and no resolvable text, so the emitter would drop it and leave the string unterminated. A multiline string closes with the same delimiter it opens with, so the external close emits the opener’s literal. Derived purely from grammar structure (the STRING-open / external-close envelope); stays in the generic emitter.

Implementations

impl Grammar

pub fn from_bytes(protocol: &str, bytes: &[u8]) -> Result<Self, ParseError>

Parse a grammar’s grammar.json bytes.

Builds the subtyping closure as part of construction so every downstream lookup is O(1). The closure is the least relation containing (K, K) for every rule key K and closed under:

• hidden-rule expansion: if S is hidden and a SYMBOL S may reach SYMBOL K, then K ⊑ S.
• supertype expansion: if S is in the grammar’s supertypes block and K is one of S’s alternatives, then K ⊑ S.
• alias renaming: if a rule body contains ALIAS { content: SYMBOL R, value: A, named: true } where R reaches kind K (or K = R when no further hop), then A ⊑ R and K ⊑ A.

Errors

Returns ParseError::EmitFailed when the bytes are not a valid grammar.json document.

pub fn from_bytes_with_node_types( protocol: &str, grammar_bytes: &[u8], node_types_bytes: Option<&[u8]>, ) -> Result<Self, ParseError>

Parse a grammar from both grammar.json and optionally node-types.json bytes.

Errors

Returns ParseError::EmitFailed when grammar_bytes is not a valid grammar.json document.

Trait Implementations

impl Clone for Grammar

fn clone(&self) -> Grammar

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Grammar

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Grammar

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.