Struct hime_redist::lexers::impls::ContextFreeLexer
source · pub struct ContextFreeLexer<'a> { /* private fields */ }Expand description
Represents a context-free lexer (lexing rules do not depend on the context)
Implementations§
source§impl<'a> ContextFreeLexer<'a>
impl<'a> ContextFreeLexer<'a>
sourcepub fn new(
repository: TokenRepository<'a>,
errors: &'a mut ParseErrors,
automaton: Automaton,
separator_id: u32
) -> ContextFreeLexer<'a>
pub fn new(
repository: TokenRepository<'a>,
errors: &'a mut ParseErrors,
automaton: Automaton,
separator_id: u32
) -> ContextFreeLexer<'a>
Creates a new lexer
Trait Implementations§
source§impl<'a> Lexer<'a> for ContextFreeLexer<'a>
impl<'a> Lexer<'a> for ContextFreeLexer<'a>
source§fn get_terminals(&self) -> &'static [Symbol]
fn get_terminals(&self) -> &'static [Symbol]
Gets the terminals matched by this lexer
source§fn get_output(&self) -> &TokenRepository<'a>
fn get_output(&self) -> &TokenRepository<'a>
Gets the lexer’s output stream of tokens
source§fn get_errors(&mut self) -> &mut ParseErrors
fn get_errors(&mut self) -> &mut ParseErrors
Gets the lexer’s errors
source§fn get_recovery_distance(&self) -> usize
fn get_recovery_distance(&self) -> usize
Gets the maximum Levenshtein distance to go to for the recovery of a matching failure. A distance of 0 indicates no recovery.
source§fn set_recovery_distance(&mut self, distance: usize)
fn set_recovery_distance(&mut self, distance: usize)
Sets the maximum Levenshtein distance to go to for the recovery of a matching failure. A distance of 0 indicates no recovery.
source§fn get_next_token(
&mut self,
_contexts: &dyn ContextProvider
) -> Option<TokenKernel>
fn get_next_token(
&mut self,
_contexts: &dyn ContextProvider
) -> Option<TokenKernel>
Gets the next token in the input