Struct tokenizer_lib::GeneratorTokenQueue [−][src]
pub struct GeneratorTokenQueue<T, TData, TGeneratorState, TGenerator> where
T: PartialEq,
for<'a> TGenerator: FnMut(&mut TGeneratorState, &mut GeneratorTokenQueueBuffer<'a, T, TData>), { /* fields omitted */ }
Expand description
A token queue which has a backing generator/lexer which is called when needed by parsing logic
Implementations
impl<T, TData, TGeneratorState, TGenerator> GeneratorTokenQueue<T, TData, TGeneratorState, TGenerator> where
T: PartialEq,
for<'a> TGenerator: FnMut(&mut TGeneratorState, &mut GeneratorTokenQueueBuffer<'a, T, TData>),
impl<T, TData, TGeneratorState, TGenerator> GeneratorTokenQueue<T, TData, TGeneratorState, TGenerator> where
T: PartialEq,
for<'a> TGenerator: FnMut(&mut TGeneratorState, &mut GeneratorTokenQueueBuffer<'a, T, TData>),
Create a new GeneratorTokenQueue
with a lexer function and initial state
Trait Implementations
impl<T, TData, TGeneratorState, TGenerator> TokenReader<T, TData> for GeneratorTokenQueue<T, TData, TGeneratorState, TGenerator> where
T: PartialEq + Debug,
TData: Debug,
for<'a> TGenerator: FnMut(&mut TGeneratorState, &mut GeneratorTokenQueueBuffer<'a, T, TData>),
impl<T, TData, TGeneratorState, TGenerator> TokenReader<T, TData> for GeneratorTokenQueue<T, TData, TGeneratorState, TGenerator> where
T: PartialEq + Debug,
TData: Debug,
for<'a> TGenerator: FnMut(&mut TGeneratorState, &mut GeneratorTokenQueueBuffer<'a, T, TData>),
Returns a reference to next token but does not advance current position
Runs the closure (cb) over upcoming tokens. Passes the value behind the Token to the closure. Will stop and return a reference to the next Token from when the closure returns true. Returns None if scanning finishes before closure returns true. Does not advance the reader. Read more
Tests that next token matches an expected type. Will return error if does not
match. The Ok
value contains the data of the valid token.
Else it will return the Err with the expected token type and the token that did not match Read more