Trait lrpar::lex::Lexer

pub trait Lexer<StorageT: Hash + PrimInt + Unsigned> {
    fn next(&mut self) -> Option<Result<Lexeme<StorageT>, LexError>>;
    fn lexeme_str(&self, _: &Lexeme<StorageT>) -> &str;
    fn offset_line_col(&self, _: usize) -> (usize, usize);

    fn all_lexemes(&mut self) -> Result<Vec<Lexeme<StorageT>>, LexError> { ... }
}

Roughly speaking, Lexer is an iterator which collectively produces Lexemes, as well as collecting the newlines encountered so that it can later optionally answer queries of the form “what’s the line and column number of lexeme L”.

Required Methods

fn next(&mut self) -> Option<Result<Lexeme<StorageT>, LexError>>

Return the next Lexeme in the input or a LexError. Returns None if the input has been fully lexed (or if an error occurred which prevents further lexing).

fn lexeme_str(&self, _: &Lexeme<StorageT>) -> &str

Return the user input associated with a lexeme. Panics if the lexeme is invalid (i.e. was not produced by next()).

fn offset_line_col(&self, _: usize) -> (usize, usize)

Return the line and column number of a given offset. Panics if the offset exceeds the known input.

Provided Methods

fn all_lexemes(&mut self) -> Result<Vec<Lexeme<StorageT>>, LexError>

Return all this lexer’s remaining lexemes or a LexError if there was a problem when lexing.

Implementors