Struct Lexer 

pub struct Lexer {
    pub line: usize,
    pub suppress_m_regex: u32,
    pub last_was_bare_positional: bool,
    pub bare_positional_indices: HashSet<usize>,
    /* private fields */
}

Fields

line: usize

suppress_m_regex: u32

When > 0, the lexer treats m followed by / as a plain identifier instead of m// regex syntax. Used in thread/pipeline stages where /m/ should be a regex grep filter, not m//.

last_was_bare_positional: bool

Set true by Self::next_token right before returning a token that originated from a bare positional alias (_, _0, _1, …) — i.e. without a leading $ sigil. Read by Self::tokenize to record the emitted token’s index in Self::bare_positional_indices. Reset to false at the top of every next_token call.

bare_positional_indices: HashSet<usize>

Indices into the token vector returned by Self::tokenize for every bare-positional token. Used by the parser’s my $X = EXPR rule to auto-wrap an RHS that contains free positional aliases into an implicit zero-arg coderef (so my $f = _ * 2 ≡ my $f = fn { _ * 2 }).

Implementations

impl Lexer

pub fn new(input: &str) -> Self

pub fn new_with_file(input: &str, file: impl Into<String>) -> Self

pub fn next_token(&mut self) -> PerlResult<Token>

pub fn tokenize(&mut self) -> PerlResult<Vec<(Token, usize)>>

Tokenize entire input.