Struct ASTNode 

#[non_exhaustive]
pub struct ASTNode {
    pub children: Option<Vec<ASTNode>>,
    pub token_count: u32,
    pub text: u32,
}

AST node. Grammar rules have children, tokens do not.

The total structure of the AST is defined by the grammar that it was parsed with.

Why isn’t this an enum? Option<Vec<ASTNode>> and other two-variant enums containing a Vec undergo niche optimization. If this were enum ASTNode { Rule{...}, Token(u32) } then it would look like you can just add a third variant (e.g. poisoned) without issue. However, doing that would actually increase the size of the ASTNode from 32 bytes to 40 bytes.

If the size is ever forced above 32 bytes (e.g. increasing token count from u32 to u64) then I’ll probably change it to an enum.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

children: Option<Vec<ASTNode>>

If Some, this node is a parent/nonterminal. If None, this node is a token/leaf/terminal.

token_count: u32

Due to error recovery, AST nodes can be marked as “poisoned”.

When a node is poisoned, its token count is XOR’d with !0u32 (all one-bits).

text: u32

Index into grammar.string_cache_inv, giving an Rc<String>.

For parents, it’s the name of the associated grammar rule.

For tokens, it’s the token content (the actual token contents, but what it was matched with, i.e. it’s not a regex).

Implementations

impl ASTNode

pub fn new(children: Option<Vec<ASTNode>>, token_count: u32, text: u32) -> Self

pub fn is_poisoned(&self) -> bool

pub fn get_real_token_count(&self) -> u32

Trait Implementations

impl Clone for ASTNode

fn clone(&self) -> ASTNode

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ASTNode

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ASTNode

fn default() -> ASTNode

Returns the “default value” for a type.