Module token_stream_reader 

Function reader.next() returns Option<Result<Token, AsonError>>.

Returning Option::None indicates the end of the stream, while Some(Err) indicates a lexing error, and Some(Ok) contains the next token.

This reader does not perform any parsing or validation of the token stream, it simply reads and returns tokens as they are lexed and normalized.

Note that the Token::_Plus and Token::_Minus tokens wouldn’t be returned because they are normalized into signed numbers by NormalizeSignedNumberIter.

Structs

TokenStreamReader

Functions

stream_from_char_iterator

stream_from_reader

stream_from_str