Trait tokenizer_lib::TokenReader[−][src]

pub trait TokenReader<T: PartialEq, TData> {
    fn peek(&self) -> Option<&Token<T, TData>>;

    fn next(&mut self) -> Option<Token<T, TData>>;

    fn scan(&self, f: impl FnMut(&T) -> bool) -> Option<&Token<T, TData>>;

    fn expect_next(
        &mut self, 
        expected_type: T
    ) -> Result<TData, Option<(T, Token<T, TData>)>> { ... }
}

Trait for a reader which returns tokens over a current sequence

Required methods

fn peek(&self) -> Option<&Token<T, TData>>[src]

Returns reference to next token but does not advance iterator forward

fn next(&mut self) -> Option<Token<T, TData>>[src]

Returns token and advances forward

fn scan(&self, f: impl FnMut(&T) -> bool) -> Option<&Token<T, TData>>[src]

Runs the closure over the upcoming tokens. Passes the value behind the Token to the closure. Will stop and return a reference to the next Token from when the closure returns true. Returns None if reader finishes before closure returns true. Does not advance the reader.

Used for lookahead and then doing branching based on return value during parsing

Loading content...

Provided methods

fn expect_next(
    &mut self,
    expected_type: T
) -> Result<TData, Option<(T, Token<T, TData>)>>[src]

Tests that next token matches an expected type. Will return error if does not match. The ok value contains the data of the valid token. Else it will return the Err with the expected token type and the token that did not match

Loading content...

Implementors

impl<T: PartialEq, TData> TokenReader<T, TData> for StreamedTokenReader<T, TData>[src]

impl<T: PartialEq, TData> TokenReader<T, TData> for StaticTokenChannel<T, TData>[src]

Loading content...