Struct Lexer

pub struct Lexer<T>
where
    T: Read,
{ /* private fields */ }

This structure holds the data of the actual lexer. It is expected that this structure can be mutated by the next_token function, which advances to the next token, and returns the term found by the lexer.

Implementations

impl<T: Read> Lexer<T>

The lexer implementation allows to read the tokens from a given type implementing the Read trait. If an error occurs while reading, or EOT has been reached, the next_token function will return Error or EOT as its term.

pub fn create(input: T) -> Self

Creates a new lexer from the given input, where input supports the Read trait.

Trait Implementations

impl<T: Read> LexerBase for Lexer<T>

fn next_token(&mut self) -> Token

Finds the next token in a mutable self. If the next token can be found, returns it into Token. If the file or stream of data is found, the special term EOT is returned. If an error occurred of some sort, the special term Error is returned.

fn current_stream(&self) -> Option<String>

Returns the current stream’s name, if any (for example, the normal lexer does know this value)