Struct JsonStream

pub struct JsonStream<T, S> { /* private fields */ }

A Stream implementation that can be used to parse Newline Delimited JSON values from a byte stream. It does so by buffering bytes internally and parsing them as they are received. This means that the stream will not yield values until a full JSON value has been received.

After a full JSON value has been parsed and yielded, the stream will delete the bytes that were used to parse the value from the internal buffer. This means that the stream will not use more memory than is necessary to contain the maximum buffer size specified) as well as any JSON values previously parsed but not yet yielded.

Implementations

impl<T, S: Unpin> JsonStream<T, S>

pub fn new(stream: S) -> Self

Create a new JsonStream with the default buffer capacity.

pub fn new_with_max_capacity(stream: S, max_capacity: usize) -> Self

Create a new JsonStream with a custom maximum buffer capacity.

The maximum buffer capacity is the maximum amount of bytes that will be buffered before the stream is terminated. This is to prevent malformed streams from causing the server to run out of memory.

As a rule of thumb, this number should be at least as large as the largest entry in the stream. Additionally, it’s best if it’s a power of 2 minus 1 (e.g. 1023, 2047, 4095, etc.) as this allows the internal buffer to be more efficient. This is not a requirement, however.

Lastly, it is not guaranteed that this is the maximum amount of memory that will be used by the stream. This is because the internal buffer may allocate 2x the amount of bytes specified as well as waste some space in the buffer.

pub fn set_max_buffer_size(&mut self, max_capacity: usize)

Controls how large the internal buffer can grow in bytes. If the buffer grows larger than this the stream is terminated as it is assumed that the stream is malformed. If this number is too large, a malformed stream can cause the server to run out of memory.

As a rule of thumb, this number should be at least as large as the largest entry in the stream.

The default value is 8 MB.

Trait Implementations

impl<T, S, B, E> Stream for JsonStream<T, S>

where T: DeserializeOwned, B: Deref<Target = [u8]>, S: Stream<Item = Result<B, E>> + Unpin,

type Item = Result<T, E>

Values yielded by the stream.

fn poll_next( self: Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll<Option<Self::Item>>

Attempt to pull out the next value of this stream, registering the current task for wakeup if the value is not yet available, and returning None if the stream is exhausted.

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the stream.

impl<'pin, T, S> Unpin for JsonStream<T, S>

where PinnedFieldsOf<__JsonStream<'pin, T, S>>: Unpin,