Struct rustemo::StringLexer

pub struct StringLexer<C, S, TK, TR: 'static, const TERMINAL_COUNT: usize> { /* private fields */ }

A lexer that operates over string inputs and uses generated string and regex recognizers provided by the parser table.

Implementations

impl<'i, C: Context<'i, str, S, TK>, S: State, TK, TR: TokenRecognizer<'i>, const TERMINAL_COUNT: usize> StringLexer<C, S, TK, TR, TERMINAL_COUNT>

pub fn new( skip_ws: bool, token_recognizers: &'static [TR; TERMINAL_COUNT] ) -> Self

Trait Implementations

impl<'i, C, S, TK, TR, const TERMINAL_COUNT: usize> Lexer<'i, C, S, TK> for StringLexer<C, S, TK, TR, TERMINAL_COUNT>

where C: Context<'i, str, S, TK>, S: State + Into<usize>, TK: Debug + Into<usize> + Copy + 'i, TR: TokenRecognizer<'i>,

type Input = str

fn next_tokens( &self, context: &mut C, input: &'i Self::Input, expected_tokens: Vec<(TK, bool)> ) -> Box<dyn Iterator<Item = Token<'i, Self::Input, TK>> + 'i>

Given the current context, this method yield an iterator over possible tokens found at the current location where the order and kinds of token to look for, and its finish flags, are given by the expected_tokens parameter.