Struct Lexer 

pub struct Lexer<'a> { /* private fields */ }

Lexer for regex patterns.

Implementations

impl<'a> Lexer<'a>

pub fn new(input: &'a str) -> Self

Create a new lexer for the given input.

pub fn new_with_flags(input: &'a str, verbose: bool) -> Self

Create a new lexer with verbose mode.

pub fn position(&self) -> usize

Get the current position in the input.

pub fn set_position(&mut self, pos: usize)

Set the position in the input (for backtracking).

pub fn try_match(&mut self, s: &str) -> bool

Try to match a specific string, advancing if successful.

This is useful for parser extensions that need to match multi-character sequences like keywords or specific syntax.

pub fn next_token(&mut self) -> Result<Token>

Get the next token.

Errors

Returns an error if an invalid escape sequence is encountered.

pub fn peek_token(&mut self) -> Result<Token>

Peek at the next token without consuming it.

Errors

Returns an error if an invalid escape sequence is encountered.

pub fn is_eof(&mut self) -> bool

Check if we’ve reached the end of input.

pub fn remaining(&self) -> &'a str

Get remaining input from current position.