Struct Lexer 

pub struct Lexer<TokenType> { /* private fields */ }

Represents a finished Lexer

Implementations

impl<TokenType> Lexer<TokenType>

pub fn init(&mut self, data: String)

Resets the parser to the starting state with input data

pub fn tok(&mut self, skip_ws: bool) -> Option<TokenType>

Returns the next Token, or None if no token is found

pub fn is_eof(&self) -> bool

Returns true if the end of input has been reached.