Struct Lexer 

pub struct Lexer<'src> {
    pub errors: Vec<LexerError>,
    /* private fields */
}

Streaming PHP lexer with two-token lookahead.

Fields

errors: Vec<LexerError>

Errors collected so far.

Implementations

impl<'src> Lexer<'src>

pub fn new(source: &'src str) -> Self

Creates a lexer for source, starting in inline-HTML mode.

Spans hold u32 offsets, so a source over 4 GiB is not lexed at all: the lexer yields only Eof and records a LexerErrorKind::FileTooLarge error.

pub fn new_at(source: &'src str, offset: usize) -> Self

Create a lexer starting in PHP mode at a given byte offset within source. The caller guarantees that source[offset..] contains valid PHP expression content (no <?php tag needed — the lexer is pre-set to PHP mode). Spans produced will be correct absolute offsets into source.

Like Lexer::new, a source over 4 GiB yields only Eof and an error.

pub fn is_verbatim_string(&mut self, start: u32) -> bool

Whether the single- or double-quoted string token starting at start contains no escape sequence and, for a double-quoted string, no $, so its contents between the quotes can be used verbatim without decoding.

Only meaningful for string tokens this lexer has already produced.

pub fn source(&self) -> &'src str

The source text being lexed.

pub fn peek(&mut self) -> &Token

Returns the next token without consuming it.

pub fn peek2(&mut self) -> &Token

Peek two tokens ahead (past the next token).

pub fn next_token(&mut self) -> Token

Consumes and returns the next token.

pub fn token_text(&self, token: &Token) -> &'src str

Get the text slice corresponding to a token