Module ndjson 

Available on crate feature worker-batch and (crate features worker-batch or worker-pool or worker) only.

NDJSON (newline-delimited JSON) batch processing utilities.

Splits NDJSON byte payloads into individual lines and optionally parses them in parallel via AdaptiveWorkerPool::process_batch.

Does NOT depend on a specific JSON parser – the parse function is a closure. Use with sonic-rs, serde_json, or any other parser.

Example

use hyperi_rustlib::worker::ndjson;

let payload = b"{\"a\":1}\n{\"b\":2}\n{\"c\":3}\n";
let lines = ndjson::split_lines(payload);
assert_eq!(lines.len(), 3);

// Parallel parse (with worker pool)
let parsed = pool.process_batch(&lines, |line| {
    sonic_rs::from_slice::<Value>(line).map_err(|e| e.to_string())
});

Functions

count_lines

Count the number of NDJSON lines in a payload without allocating.

split_lines

Split an NDJSON payload into individual line slices.