Struct JsonTokenizer

pub struct JsonTokenizer<I: Iterator<Item = u8>> { /* private fields */ }

A pull-based tokenizer which takes an iterator over bytes and emits JsonTokens.

Implementations

impl<I: Iterator<Item = u8>> JsonTokenizer<I>

pub fn new(it: I) -> Self

Create a new JsonTokenizer

pub fn location(&self) -> Location

The location after the end of the last token that was returned from next_token().

pub fn expect_eof(&mut self) -> Result<(), JsonParseError>

Returns an error if there is more than just white space in the remaining bytes.

pub fn next_token(&mut self) -> JsonParseResult<JsonToken>

Parses a token and returns it, or an error.

pub fn next_token_and_location( &mut self, ) -> JsonParseResult<(JsonToken, Location)>

Parses a token and returns it along with its location, or an error.