Struct Lexer 

pub struct Lexer {
    pub source_string: String,
    pub source_chars: Vec<char>,
    pub start: usize,
    pub current: usize,
    pub tokens: Vec<Token>,
    pub len: usize,
    pub has_errors: bool,
}

Fields

source_string: String

source_chars: Vec<char>

start: usize

current: usize

tokens: Vec<Token>

len: usize

has_errors: bool

Implementations

impl Lexer

pub fn new(input: &String) -> Self

Constructor for lexer

Arguments

• input - Input string to tokenize.

Returns

• Lexer - new instance of lexer with prepopulated fields.

pub fn scan(&mut self)

Top level public function to start tokenizing the input string.

pub fn next_token(&mut self) -> Token

returns the next token, and also consumes it.

pub fn peek(&self) -> Token

returns the next token, but doesn’t consume it.