Struct TokenStream 

pub struct TokenStream { /* private fields */ }

A lightweight wrapper around a Vec<Token> that provides cursor-based traversal without requiring a mutable Parser.

Implementations

impl TokenStream

pub fn new(tokens: Vec<Token>) -> Self

Create a new token stream from a token vector.

pub fn from_src(src: &str) -> Self

Lex src and wrap the result.

pub fn peek(&self) -> Option<&Token>

Peek at the current token without consuming it.

pub fn next(&mut self) -> Option<&Token>

Advance and return the current token.

pub fn is_empty(&self) -> bool

Whether the stream is exhausted.

pub fn remaining(&self) -> usize

Number of remaining tokens.

pub fn total_len(&self) -> usize

Total number of tokens (including consumed ones).

pub fn reset(&mut self)

Reset the cursor to the beginning.

pub fn collect_remaining(&self) -> Vec<&Token>

Collect remaining tokens into a Vec.

Trait Implementations

impl Clone for TokenStream

fn clone(&self) -> TokenStream

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenStream

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.