tokens_in

Function tokens_in 

Source
pub fn tokens_in<'a, Token: 'a, Tokenizer, StrIter, Str>(
    iter: StrIter,
    tokenizer: &'a Tokenizer,
) -> impl Iterator<Item = (usize, Range<usize>, Token)> + 'a
where StrIter: Iterator<Item = Str> + 'a, Str: Into<&'a str> + 'a, Tokenizer: Fn(&mut Lexer<'a>) -> Option<Token>,
Expand description

Returns an Iterator over the tokens found in the specified lines, along with their line numbers and column ranges (both of which start at 0).