Struct LexedStream 

pub struct LexedStream<'lexer, 'stream> { /* private fields */ }

Summary

LexedStream is the interface used to tokenize a stream. To do so, you first create a Lexer which defines the lexing grammar. You then create the LexedStream with an exclusive reference to a [StringStream][beans::stream::StringStream]. Then, you get the tokens through LexedStream, which will consume the stream but only for as much as you require. This allows you to use a Lexer for many [StringStream][beans::stream::StringStream], and conversely to tokenize a single [StringStream][beans::stream::StringStream] with different Lexer.

Implementations

impl<'lexer, 'stream> LexedStream<'lexer, 'stream>

pub fn new(lexer: &'lexer Lexer, stream: &'stream mut StringStream) -> Self

Create a new LexedStream instance.

pub fn last_span(&self) -> &Span

Get the last span lexed. Useful if you want to know where you failed to find a token.

impl LexedStream<'_, '_>

pub fn next_any(&mut self) -> Result<Option<&Token>>

Lex any token.

pub fn next(&mut self, allowed: Allowed) -> Result<Option<&Token>>

Lex any allowed token.

pub fn peek(&self) -> Option<&Token>

Peek for the most recently lexed token, which has not been droped.

pub fn drop_last(&mut self)

Drop the last token.

pub fn lexer(&self) -> &Lexer

Get the lexer

pub fn is_empty(&self) -> bool

Trait Implementations

impl<'lexer, 'stream> Debug for LexedStream<'lexer, 'stream>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.