Struct JsonArrayToNdjsonReader 

pub struct JsonArrayToNdjsonReader<R: Read> { /* private fields */ }

A streaming reader that converts JSON array format to NDJSON format.

This reader wraps an underlying reader containing JSON array data [{...}, {...}, ...] and transforms it on-the-fly to newline-delimited JSON format that Arrow’s JSON reader can process.

Implements both Read and BufRead traits for compatibility with Arrow’s ReaderBuilder::build() which requires BufRead.

Transformation Rules

• Skip leading [ and whitespace before it
• Convert top-level , (between objects) to \n
• Skip whitespace at top level (between objects)
• Stop at trailing ]
• Preserve everything inside objects (including nested [, ], ,)
• Properly handle strings (ignore special chars inside quotes)

Example

Input:  [{"a":1}, {"b":[1,2]}, {"c":"x,y"}]
Output: {"a":1}
        {"b":[1,2]}
        {"c":"x,y"}

Implementations

impl<R: Read> JsonArrayToNdjsonReader<R>

pub fn new(reader: R) -> Self

Create a new streaming reader that converts JSON array to NDJSON.

pub fn with_capacity(reader: R, capacity: usize) -> Self

Create a new streaming reader with custom buffer size.

Larger buffers improve throughput but use more memory. Total memory usage is approximately 2 * capacity (input + output buffers).

pub fn validate_complete(&self) -> Result<()>

Check if the JSON array was properly terminated.

This should be called after all data has been read.

Returns an error if:

• Unbalanced braces/brackets (depth != 0)
• Unterminated string
• Missing closing ]
• Unexpected trailing content after ]

Trait Implementations

impl<R: Read> BufRead for JsonArrayToNdjsonReader<R>

fn fill_buf(&mut self) -> Result<&[u8]>

Returns the contents of the internal buffer, filling it with more data, via Read methods, if empty.

fn consume(&mut self, amt: usize)

Marks the given amount of additional bytes from the internal buffer as having been read. Subsequent calls to read only return bytes that have not been marked as read.

fn has_data_left(&mut self) -> Result<bool, Error>

🔬This is a nightly-only experimental API. (buf_read_has_data_left)

Checks if there is any data left to be read.

fn read_until(&mut self, byte: u8, buf: &mut Vec<u8>) -> Result<usize, Error>

Reads all bytes into buf until the delimiter byte or EOF is reached.

fn skip_until(&mut self, byte: u8) -> Result<usize, Error>

Skips all bytes until the delimiter byte or EOF is reached.

fn read_line(&mut self, buf: &mut String) -> Result<usize, Error>

Reads all bytes until a newline (the 0xA byte) is reached, and append them to the provided String buffer.

fn split(self, byte: u8) -> Split<Self>

where Self: Sized,

Returns an iterator over the contents of this reader split on the byte byte.

fn lines(self) -> Lines<Self>

where Self: Sized,

Returns an iterator over the lines of this reader.

impl<R: Read> Read for JsonArrayToNdjsonReader<R>

fn read(&mut self, buf: &mut [u8]) -> Result<usize>

Pull some bytes from this source into the specified buffer, returning how many bytes were read.

fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize, Error>

Like read, except that it reads into a slice of buffers.

fn is_read_vectored(&self) -> bool

🔬This is a nightly-only experimental API. (can_vector)

Determines if this Reader has an efficient read_vectored implementation.

fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize, Error>

Reads all bytes until EOF in this source, placing them into buf.

fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>

Reads all bytes until EOF in this source, appending them to buf.

fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Error>

Reads the exact number of bytes required to fill buf.

fn read_buf(&mut self, buf: BorrowedCursor<'_>) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Pull some bytes from this source into the specified buffer.

fn read_buf_exact(&mut self, cursor: BorrowedCursor<'_>) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Reads the exact number of bytes required to fill cursor.

fn by_ref(&mut self) -> &mut Self

where Self: Sized,

Creates a “by reference” adapter for this instance of Read.

fn bytes(self) -> Bytes<Self>

where Self: Sized,

Transforms this Read instance to an Iterator over its bytes.

fn chain<R>(self, next: R) -> Chain<Self, R>

where R: Read, Self: Sized,

Creates an adapter which will chain this stream with another.

fn take(self, limit: u64) -> Take<Self>

where Self: Sized,

Creates an adapter which will read at most limit bytes from it.

fn read_array<const N: usize>(&mut self) -> Result<[u8; N], Error>

where Self: Sized,

🔬This is a nightly-only experimental API. (read_array)

Read and return a fixed array of bytes from this source.