Expand description
The shape of the generated rule table, and the filter that reads it.
generated::rules is data. This is the handful of types that give it meaning, and they are
written by hand because they are an interface: the generator in xtask writes discriminants
that have to mean the same thing here, and there is a test on each side that says so.
The one idea worth stating on its own is the FIRST filter. A choice in this grammar can have
forty alternatives, Statement has thirty six, and upstream tries them in order, descending
into each one far enough to fail. Every node here carries a 64 bit set of the token keys it can
begin with, and a token maps to exactly one of those keys, so an alternative that cannot
possibly match is skipped on one AND rather than on a subtree walk. The set is a superset by
construction: keywords share 58 buckets, so a bit that is set may still fail, and a bit that is
clear can never match. Being wrong in that direction costs a wasted attempt and never changes
what the parser accepts, which is what makes it safe to put in front of a dialect we are
copying rather than defining.
spec/20-the-grammar.md sections 3 and 5.
Structs§
Enums§
- Op
- What a node is.
- Suggestion
- What an identifier matcher was built to suggest.
Constants§
- BUCKETS
- The keyword buckets, being the rest of the 64.
- FIRST_
ANY_ KEYWORD - Every keyword bucket at once, which is what an identifier matcher accepts, because which words it takes depends on the class and on the position and the filter is not the place to decide it.
- FIRST_
END - The end of the input.
- FIRST_
IDENT - A bare or quoted name.
- FIRST_
NUMBER - A numeric literal.
- FIRST_
OPERATOR - An operator or a piece of punctuation.
- FIRST_
STRING - A string literal.
- FIRST_
TERMINATOR - A
;. - KIND_
BITS - How many bits of a FIRST set go to token kinds before the keyword buckets start.