Struct NdjsonDecoder 

pub struct NdjsonDecoder { /* private fields */ }

An incremental NDJSON framer: push bytes, take complete lines.

NdjsonStream drives one over a byte stream; use it directly when the bytes do not come as a Stream (a callback API, a test).

Framing is linear: each byte is scanned for a line ending once, however the chunks split the lines. A size cap (max_line_size) bounds the line still waiting for its ending, where memory grows from chunk to chunk.

use allama_http::ndjson::NdjsonDecoder;

let mut decoder = NdjsonDecoder::new();
decoder.push(b"{\"a\":1}\r");
assert_eq!(decoder.next_line(), Ok(None));
decoder.push(b"\n\n{\"b\":");
assert_eq!(decoder.next_line(), Ok(Some(r#"{"a":1}"#.to_owned())));
assert_eq!(decoder.next_line(), Ok(None));
decoder.push(b"2}");
// The body ended: the last line needs no line ending.
assert_eq!(decoder.finish(), Ok(Some(r#"{"b":2}"#.to_owned())));

Implementations

impl NdjsonDecoder

pub const DEFAULT_MAX_LINE_SIZE: usize

The default max_line_size: 64 MiB, as SseDecoder::DEFAULT_MAX_EVENT_SIZE, for the same reasons: a final line can repeat the whole response, and streamed base64 images are large.

pub fn new() -> Self

A framer at the start of a stream, with the default size cap.

pub fn max_line_size(self, limit: usize) -> Self

Set the size cap, in bytes (default DEFAULT_MAX_LINE_SIZE).

It bounds a line, its line ending excluded. push checks the line still waiting for its ending before it buffers a chunk, so that line never grows much past the cap.

Past the cap, next_line fails with NdjsonError::LineTooLarge, and framing cannot go on: the framer drops what it holds, ignores later bytes and keeps returning the error.

use allama_http::ndjson::{NdjsonDecoder, NdjsonError};

let mut decoder = NdjsonDecoder::new().max_line_size(8);
decoder.push(b"{\"a\":12}");
assert_eq!(decoder.next_line(), Ok(None));
// A 9th byte without a line ending.
decoder.push(b"3");
assert!(matches!(
    decoder.next_line(),
    Err(NdjsonError::LineTooLarge { limit: 8, .. })
));

pub fn push(&mut self, bytes: &[u8])

Append received bytes. Chunk boundaries may fall anywhere, inside a line, a CRLF or a UTF-8 sequence.

Bytes that would take the line still waiting for its ending past the size cap are not buffered: next_line returns the lines completed before that line, then fails.

pub fn next_line(&mut self) -> Result<Option<String>, NdjsonError<Infallible>>

The next complete, non-blank line, without its line ending, or None until more bytes arrive.

Errors

• NdjsonError::InvalidUtf8 when a line is not valid UTF-8. The line is skipped, and framing can go on, but the stream is corrupt: NdjsonStream ends there.
• NdjsonError::LineTooLarge past the size cap. Framing cannot go on.

A framer fails with no other error: it reads no transport.

pub fn finish(&mut self) -> Result<Option<String>, NdjsonError<Infallible>>

The last line, once the body ended: what is left without a line ending, unless it is blank. Call it after next_line returned None; the framer is empty afterwards.

Errors

As next_line.

Trait Implementations

impl Debug for NdjsonDecoder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for NdjsonDecoder

fn default() -> Self

Returns the “default value” for a type.