Struct Lexer 

pub struct Lexer<'src> { /* private fields */ }

Streaming PHP lexer with two-token lookahead.

Implementations

impl<'src> Lexer<'src>

pub fn new(source: &'src str) -> Self

Creates a lexer for source, starting in inline-HTML mode.

Spans hold u32 offsets, so a source over 4 GiB is not lexed at all: the lexer yields only Eof and records a LexerErrorKind::FileTooLarge error.

pub fn new_at(source: &'src str, offset: usize) -> Self

Create a lexer starting in PHP mode at a given byte offset within source. The caller guarantees that source[offset..] contains valid PHP expression content (no <?php tag needed — the lexer is pre-set to PHP mode). Spans produced will be correct absolute offsets into source.

Like Lexer::new, a source over 4 GiB yields only Eof and an error.

pub fn with_attributes(self, enabled: bool) -> Self

Sets whether #[ opens an attribute, as from PHP 8.0, or starts a # comment. Defaults to true; set it before the first token is read.

pub fn errors(&self) -> &[LexerError]

Errors collected so far.

pub fn into_errors(self) -> Vec<LexerError>

Consumes the lexer, returning the errors collected so far.

pub fn source(&self) -> &'src str

The source text being lexed.

pub fn peek(&mut self) -> &Token

Returns the next token without consuming it.

pub fn peek2(&mut self) -> &Token

Peek two tokens ahead (past the next token).

pub fn next_token(&mut self) -> Token

Consumes and returns the next token.

pub fn token_text(&self, token: &Token) -> &'src str

Get the text slice corresponding to a token