Struct rustlr::lexer_interface::TerminalToken

pub struct TerminalToken<'t, AT: Default> {
    pub sym: &'t str,
    pub value: AT,
    pub line: usize,
    pub column: usize,
}

This the token type required by Rustlr while parsing. A TerminalToken must correspond to a terminal symbol of the grammar being parsed. The sym field of the struct must correspond to the name of the terminal as defined by the grammar and the value must be of type AT, which the is abstract syntax type (valuetype) of the grammar. It also includes the starting line and column positions of the token. These tokens are generated by implementing Tokenizer::nextsym.

This struct is intended to replace Lextoken, and does not use owned strings. Currently this structure lives side-by side with Lextoken for compatibility.

Fields

sym: &'t str

value: AT

line: usize

column: usize

Implementations

impl<'t, AT: Default> TerminalToken<'t, AT>

pub fn new(s: &'t str, v: AT, ln: usize, cl: usize) -> TerminalToken<'t, AT>

creates new lexical token with sym s, value v, line ln and column cl

pub fn transfer(&self, s: &'t str, v: AT) -> TerminalToken<'t, AT>

transfers lexical information (line/column) to new TerminalToken

pub fn from_raw( rt: (RawToken<'t>, usize, usize), s: &'t str, v: AT ) -> TerminalToken<'t, AT>

transfers lexical information from a (RawToken,line,column) triple returned by StrTokenizer::next_token to a new TerminalToken with sym s and value v.

pub fn lb<T>(&self, e: T) -> LBox<T>

creates an LBox vale using lexical information contained in the token.

pub fn lba<T: 'static>(&self, e: T) -> LBox<dyn Any>

creates a LBox<dyn Any>, which allows attributes of different types to be associated with grammar symbols.

pub fn lrc<T>(&self, e: T) -> LRc<T>

similar to crate::ZCParser::lb, but creates a LRc instead of LBox

pub fn lrca<T: 'static>(&self, e: T) -> LRc<dyn Any>

similar to crate::ZCParser::lba but creates a LRc

impl<'t, AT: Default + 'static> TerminalToken<'t, AT>

pub fn raw_to_lba( rt: (RawToken<'t>, usize, usize), s: &'t str, v: AT ) -> TerminalToken<'t, LBox<dyn Any>>

creates a TerminalToken from a RawToken with value of type LBox<dyn Any>