Struct hime_redist::lexers::impls::ContextSensitiveLexer
source · pub struct ContextSensitiveLexer<'a> { /* private fields */ }
Expand description
Represents a context-sensitive lexer (lexing rules do not depend on the context)
Implementations§
source§impl<'a> ContextSensitiveLexer<'a>
impl<'a> ContextSensitiveLexer<'a>
sourcepub fn new(
repository: TokenRepository<'a>,
errors: &'a mut ParseErrors,
automaton: Automaton,
separator_id: u32
) -> ContextSensitiveLexer<'a>
pub fn new(
repository: TokenRepository<'a>,
errors: &'a mut ParseErrors,
automaton: Automaton,
separator_id: u32
) -> ContextSensitiveLexer<'a>
Creates a new lexer
Trait Implementations§
source§impl<'a> Lexer<'a> for ContextSensitiveLexer<'a>
impl<'a> Lexer<'a> for ContextSensitiveLexer<'a>
source§fn get_terminals(&self) -> &'static [Symbol]
fn get_terminals(&self) -> &'static [Symbol]
Gets the terminals matched by this lexer
source§fn get_output(&self) -> &TokenRepository<'a>
fn get_output(&self) -> &TokenRepository<'a>
Gets the lexer’s output stream of tokens
source§fn get_errors(&mut self) -> &mut ParseErrors
fn get_errors(&mut self) -> &mut ParseErrors
Gets the lexer’s errors
source§fn get_recovery_distance(&self) -> usize
fn get_recovery_distance(&self) -> usize
Gets the maximum Levenshtein distance to go to for the recovery of a matching failure. A distance of 0 indicates no recovery.
source§fn set_recovery_distance(&mut self, distance: usize)
fn set_recovery_distance(&mut self, distance: usize)
Sets the maximum Levenshtein distance to go to for the recovery of a matching failure. A distance of 0 indicates no recovery.
source§fn get_next_token(
&mut self,
contexts: &dyn ContextProvider
) -> Option<TokenKernel>
fn get_next_token(
&mut self,
contexts: &dyn ContextProvider
) -> Option<TokenKernel>
Gets the next token in the input