Struct CommonTokenStream 

pub struct CommonTokenStream<S> { /* private fields */ }

Implementations

impl<S> CommonTokenStream<S>

where S: TokenSource,

pub const fn new(source: S) -> Self

Creates a token stream that filters lookahead to the default channel.

pub const fn with_channel(source: S, channel: i32) -> Self

Creates a token stream whose LT/LA operations see only channel.

pub fn fill(&mut self)

Reads tokens from the source until EOF is buffered.

pub fn get(&mut self, index: usize) -> Option<&CommonToken>

Returns the token at an absolute buffered index, fetching from the source as needed.

pub fn lt(&mut self, offset: isize) -> Option<&CommonToken>

Returns the token at one-based lookahead/lookbehind offset, skipping tokens outside the configured channel for positive offsets.

pub fn lb(&self, offset: usize) -> Option<&CommonToken>

pub const fn token_source(&self) -> &S

pub fn tokens(&self) -> &[CommonToken]

pub fn previous_visible_token_index(&mut self, index: usize) -> Option<usize>

Finds the previous buffered token visible to this stream before index.

Parser rule intervals and $text actions are defined in terms of visible tokens, but their rendered source text still includes hidden tokens between the visible start and stop. Returning the previous token on the stream channel avoids accidentally using trailing hidden whitespace as the stop token.

impl<S> CommonTokenStream<S>

where S: TokenSource,

pub fn la_token(&mut self, offset: isize) -> i32

pub fn token_type_at_index(&mut self, index: usize) -> i32

Returns the token type at a buffered absolute index, fetching from the source on demand. Past-EOF reads are reported as TOKEN_EOF so the caller does not need to special-case the buffer’s stop. The cursor is not modified, which lets hot speculative loops avoid the seek round-trip when they only need lookahead types.

pub fn next_visible_after(&mut self, index: usize) -> usize

Returns the next parser-visible token index after consuming the token at index, skipping hidden-channel tokens. The parser’s stream cursor is not modified. Used by speculative recognition that simulates token consumption thousands of times without committing it.

pub fn text(&mut self, start: usize, stop: usize) -> String

pub fn drain_source_errors(&mut self) -> Vec<TokenSourceError>

Returns and clears diagnostics emitted by the underlying token source while this stream was fetching tokens.

Trait Implementations

impl<S: Debug> Debug for CommonTokenStream<S>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<S> IntStream for CommonTokenStream<S>

where S: TokenSource,

fn consume(&mut self)

fn la(&mut self, offset: isize) -> i32

fn index(&self) -> usize

fn seek(&mut self, index: usize)

fn size(&self) -> usize

fn source_name(&self) -> &str

fn mark(&mut self) -> isize

fn release(&mut self, _marker: isize)