Struct carlotk::Tokenstream

pub struct Tokenstream { /* private fields */ }

Stores a stream of tokens.

Implementations

impl Tokenstream

pub fn from(input: &str, debug: bool) -> Self

Constructs a new token stream from an input string.

pub fn peek(&self) -> Option<Token>

Peeks at the next token in the token stream.

pub fn next(&mut self) -> Option<Token>

Gets the next token from the token stream.

pub fn next_unwrap(&mut self) -> Token

Gets the next token and unwraps it, throwing an unexpected EOF error if no token is available.

pub fn get(&mut self, class: TokenClass) -> Token

Gets the next token, unwraps it, and returns its value.

pub fn precedence(&self) -> u8

Gets the precedence of the next token.

Trait Implementations

impl Debug for Tokenstream

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.