pub struct StreamReader { /* private fields */ }Expand description
A pull reader over input that arrives in pieces: bytes go in with
feed as they arrive, events come out as soon as they are
complete.
It is the same reader as Reader, over a buffer: the
same grammar, the same budgets, and the same errors at the same offsets
(counted from the first byte ever fed), whatever the input’s chunking.
When a call needs bytes that have not arrived it returns
Step::NeedInput and changes nothing a later call would see
differently; after end_input, running out is the
ErrorKind::UnexpectedEnd that Reader reports.
Events borrow from the reader’s buffer, so each one must be dropped
before the next call. In particular, feed more input after the borrow
ends: in an else after if let Step::Ready(..) = reader.next_event()?
(Rust 2024 drops the if let temporaries before the else), or after
the loop that drains the ready events, as below; a match keeps the
event borrowed in every arm. Consumed input is dropped from the buffer as more
is fed, so a long document read with next_event as
it arrives needs memory for one token and one chunk, not for the
document. The whole-value calls (read_raw,
read_value, decode) are
different: each keeps its value’s text in the buffer until the value
ends, bounded by Limits::max_value_bytes. A token cut by a chunk
boundary is continued where it stopped, never scanned again from its
start: a string fed one byte at a time costs linear time.
The input must be UTF-8, checked as it arrives (a character may be split
across chunks). The first invalid byte is reported as
ErrorKind::InvalidUtf8 when the reader reaches it: a syntax error
earlier in the document is reported first. (A complete document given to
parse_bytes checks the encoding before the
syntax, so the two can name different errors for a document that has
both.)
Limits::max_input_bytes bounds the total fed (in
documents mode, what is fed for each document), and
Limits::max_value_bytes what a whole-value call may hold; the other
budgets apply exactly as for a complete document. After an error the
reader is finished: every later call returns the same error.
§Examples
use jsonx_lib::{Event, Step, StreamReader};
let mut reader = StreamReader::new();
let mut keys = Vec::new();
for chunk in [&b"{\"jsonr"[..], b"pc\": \"2.0\", \"i", b"d\": 12", b"}"] {
reader.feed(chunk)?;
while let Step::Ready(Some(event)) = reader.next_event()? {
if let Event::Key(key) = event {
keys.push(key.into_owned());
}
}
}
reader.end_input();
assert_eq!(reader.next_event()?, Step::Ready(None)); // the document is complete
assert_eq!(keys, ["jsonrpc", "id"]);Implementations§
Source§impl StreamReader
impl StreamReader
Sourcepub fn new() -> Self
pub fn new() -> Self
A reader with the default Limits, refusing duplicate keys in what
read_value builds. For others, use
Parser::stream_reader.
§Examples
let r = jsonx_lib::StreamReader::new();
assert_eq!(r.offset(), 0);Sourcepub fn documents(self) -> Self
pub fn documents(self) -> Self
Reads a sequence of documents instead of one, as
Reader::documents does: NDJSON, JSON
Lines, or concatenated JSON, fed as it arrives. Whole-value reads
return one document each, and Ready(None) once the input has ended
after the last.
Limits::max_input_bytes then applies to each document, not to the
stream: input is counted from where the current document began (or,
between documents, from the end of the last one), so a connection
that carries documents for as long as it stays open is never refused
for its total. A Reader, which has its whole input
already, applies the budget to all of it.
§Examples
use jsonx_lib::{Step, StreamReader};
let mut r = StreamReader::new().documents();
let mut seen = Vec::new();
for chunk in [&b"{\"id\": 1}\n{\"i"[..], b"d\": 2}\n"] {
r.feed(chunk)?;
while let Step::Ready(Some(doc)) = r.read_value()? {
seen.push(doc["id"].as_i64());
}
}
r.end_input();
assert_eq!(r.read_value()?, Step::Ready(None));
assert_eq!(seen, [Some(1), Some(2)]);Sourcepub fn feed(&mut self, bytes: &[u8]) -> Result<(), Error>
pub fn feed(&mut self, bytes: &[u8]) -> Result<(), Error>
Adds the next piece of input. A chunk may end anywhere, even inside a token or a UTF-8 character.
§Errors
ErrorKind::InputTooLarge when the input in total (in documents
mode, of the current document) grows past Limits::max_input_bytes,
or when the value a pending whole-value call is reading grows past
Limits::max_value_bytes, at the offset of the first byte over the
budget (the reader is then finished);
ErrorKind::InputAfterEnd after end_input;
any error the reader already stopped at.
§Examples
use jsonx_lib::{ErrorKind, Limits, Parser};
let mut r = Parser::new().limits(Limits::new().with_max_input_bytes(4)).stream_reader();
assert!(r.feed(b"[1,").is_ok());
assert_eq!(r.feed(b"22]").unwrap_err().kind(), ErrorKind::InputTooLarge);Sourcepub fn end_input(&mut self)
pub fn end_input(&mut self)
Says that the input has ended: nothing more will be fed. From now on, running out of input is an error, as for a complete document, and a number or whitespace at the very end is final.
§Examples
use jsonx_lib::{Event, Step, StreamReader};
let mut r = StreamReader::new();
r.feed(b"12")?;
assert_eq!(r.next_event()?, Step::NeedInput); // `123` may follow
r.end_input();
assert_eq!(r.next_event()?, Step::Ready(Some(Event::Number(12.into()))));
assert_eq!(r.next_event()?, Step::Ready(None));Sourcepub fn read_from(&mut self, source: &mut dyn Read) -> Result<usize>
Available on crate feature std only.
pub fn read_from(&mut self, source: &mut dyn Read) -> Result<usize>
std only.Reads up to 8 KiB from source and feeds it, or calls
end_input when source is at its end. Returns
the number of bytes read (0 at the end). Interrupted reads are
retried.
A budget error from feed is not an I/O error: it is
kept, and returned by the next reading call.
§Errors
Any error from source other than std::io::ErrorKind::Interrupted.
§Examples
use jsonx_lib::{Step, StreamReader};
let mut source: &[u8] = br#"{"big": [1, 2, 3]}"#;
let mut r = StreamReader::new();
let mut events = 0;
loop {
if let Step::Ready(event) = r.next_event()? {
match event {
Some(_) => events += 1,
None => break,
}
} else {
r.read_from(&mut source)?;
}
}
assert_eq!(events, 8); // {, "big", [, 1, 2, 3, ], }Sourcepub fn next_event(&mut self) -> Result<Step<Option<Event<'_>>>, Error>
pub fn next_event(&mut self) -> Result<Step<Option<Event<'_>>>, Error>
Reads the next event: Ready(Some(event)), Ready(None) once the
document is complete and the input has ended with nothing but
whitespace after it, or NeedInput.
§Errors
The first syntax error, exceeded budget, or invalid UTF-8, at its byte offset. Once an error is returned, every later call returns it.
§Examples
use jsonx_lib::{ErrorKind, Event, Step, StreamReader};
let mut r = StreamReader::new();
r.feed(b"[1,")?;
assert_eq!(r.next_event()?, Step::Ready(Some(Event::StartArray)));
assert!(r.next_event()?.is_ready()); // 1
r.feed(b"]")?;
let err = r.next_event().unwrap_err();
assert_eq!((err.kind(), err.offset()), (ErrorKind::TrailingComma, 3));Sourcepub fn read_raw(&mut self) -> Result<Step<Option<&str>>, Error>
pub fn read_raw(&mut self) -> Result<Step<Option<&str>>, Error>
Reads the next value whole and returns its exact source text, without
decoding or allocating anything, as Reader::read_raw
does, with the same positioning rules: Ready(None) when no value
comes next.
The value must be in the buffer whole before it can be returned, so
a large value costs memory for all of it, whitespace included, up to
Limits::max_value_bytes (past it, feed refuses
more). NeedInput keeps the progress: the next call continues the
same value.
After a single document’s value, a call answers as
next_event does there: NeedInput while only
whitespace has followed and the input has not ended, Ready(None)
once it has, and ErrorKind::TrailingCharacters for anything but
whitespace. finish makes the same check in one call.
§Errors
As next_event, for any error inside the value,
and ErrorKind::TrailingCharacters after the document.
§Examples
use jsonx_lib::{Event, Step, StreamReader};
let mut r = StreamReader::new();
r.feed(br#"{"id": 1, "params": {"line""#)?;
assert!(r.next_event()?.is_ready()); // {
assert!(r.next_event()?.is_ready()); // "id"
assert_eq!(r.read_raw()?, Step::Ready(Some("1")));
assert!(r.next_event()?.is_ready()); // "params"
assert_eq!(r.read_raw()?, Step::NeedInput);
r.feed(br#": 3}}"#)?;
assert_eq!(r.read_raw()?, Step::Ready(Some(r#"{"line": 3}"#)));Sourcepub fn read_value(&mut self) -> Result<Step<Option<Value<'_>>>, Error>
pub fn read_value(&mut self) -> Result<Step<Option<Value<'_>>>, Error>
Reads the next value whole and builds it as a Value, applying the
reader’s DuplicateKeys policy; positioning as for
read_raw. The value borrows from the buffer:
call Value::into_owned to keep it.
§Errors
As read_raw, plus ErrorKind::DuplicateKey
under DuplicateKeys::Error.
§Examples
use jsonx_lib::{Step, StreamReader};
let mut r = StreamReader::new();
r.feed(br#"{"a": [1, "#)?;
assert_eq!(r.read_value()?, Step::NeedInput);
r.feed(br#"2]}"#)?;
let value = r.read_value()?.ready().flatten().unwrap_or_default();
assert_eq!(value["a"][1], 2);Sourcepub fn decode<'s, T: FromJson<'s>>(
&'s mut self,
) -> Result<Step<Option<T>>, DecodeError>
pub fn decode<'s, T: FromJson<'s>>( &'s mut self, ) -> Result<Step<Option<T>>, DecodeError>
Reads the next value whole as a T (FromJson); positioning as
for read_raw. The value borrows from the buffer,
as T allows.
The value is first skipped (validated, with every budget) as it
arrives, then read as T once it is all there. Byte offsets in an
error count from the first byte fed.
§Errors
The DecodeError of T, or the reader’s error.
§Examples
use jsonx_lib::{Step, StreamReader};
let mut r = StreamReader::new().documents();
r.feed(b"[1, 2]\n[3")?;
assert_eq!(r.decode::<Vec<u8>>()?, Step::Ready(Some(vec![1, 2])));
assert_eq!(r.decode::<Vec<u8>>()?, Step::NeedInput);
r.feed(b"]\n")?;
r.end_input();
assert_eq!(r.decode::<Vec<u8>>()?, Step::Ready(Some(vec![3])));
assert_eq!(r.decode::<Vec<u8>>()?, Step::Ready(None));Sourcepub fn finish(&mut self) -> Result<Step<()>, Error>
pub fn finish(&mut self) -> Result<Step<()>, Error>
Reads to the end of the input, checking that the rest of the
document (in documents mode, of every document) is valid and that
nothing but whitespace follows: Ready(()) once the input has ended
and all of it was read, NeedInput before that. As
Reader::finish, for input that arrives in
pieces.
The whole-value calls stop at the end of the value they read, and a value nested in the document says nothing of the rest of it. Call this after the last of them to know that the input held exactly one valid document (or a sequence of valid ones).
§Errors
The first error in the rest of the input, as
next_event reports it: a syntax error, an
exceeded budget, invalid UTF-8, or ErrorKind::TrailingCharacters.
§Examples
use jsonx_lib::{ErrorKind, Step, StreamReader};
let mut r = StreamReader::new();
r.feed(br#"{"id": 1} "#)?;
assert!(r.read_value()?.is_ready());
assert_eq!(r.finish()?, Step::NeedInput); // more may follow
r.end_input();
assert_eq!(r.finish()?, Step::Ready(()));
let mut r = StreamReader::new();
r.feed(br#"{"id": 1}}"#)?;
r.end_input();
assert!(r.read_value()?.is_ready());
let err = r.finish().unwrap_err();
assert_eq!((err.kind(), err.offset()), (ErrorKind::TrailingCharacters, 9));Sourcepub fn peek_kind(&mut self) -> Result<Step<Option<Kind>>, Error>
pub fn peek_kind(&mut self) -> Result<Step<Option<Kind>>, Error>
The kind of the next value, without consuming anything: Ready(None)
when no value comes next, or when the next byte cannot start one (the
next next_event reports why).
§Errors
Any error the reader already stopped at.
§Examples
use jsonx_lib::{Kind, Step, StreamReader};
let mut r = StreamReader::new();
r.feed(b"[ ")?;
assert!(r.next_event()?.is_ready());
assert_eq!(r.peek_kind()?, Step::NeedInput);
r.feed(b"\"x\"]")?;
assert_eq!(r.peek_kind()?, Step::Ready(Some(Kind::String)));Sourcepub fn depth(&self) -> usize
pub fn depth(&self) -> usize
The number of arrays and objects open.
§Examples
let mut r = jsonx_lib::StreamReader::new();
r.feed(b"[[")?;
let _ = r.next_event()?;
let _ = r.next_event()?;
assert_eq!(r.depth(), 2);Sourcepub fn offset(&self) -> usize
pub fn offset(&self) -> usize
The byte offset, from the first byte fed, just past the input consumed so far.
§Examples
let mut r = jsonx_lib::StreamReader::new();
r.feed(b"[10, 20]")?;
let _ = r.next_event()?;
let _ = r.next_event()?;
assert_eq!(r.offset(), 3);Sourcepub fn event_offset(&self) -> usize
pub fn event_offset(&self) -> usize
The byte offset, from the first byte fed, where the most recent
event’s token began (see Reader::event_offset).
§Examples
let mut r = jsonx_lib::StreamReader::new();
r.feed(b"[ 7]")?;
let _ = r.next_event()?;
let _ = r.next_event()?;
assert_eq!(r.event_offset(), 2);