Struct ChunkProducer 

pub struct ChunkProducer<'a> { /* private fields */ }

Page-aligned chunk producer. The producer accumulates byte-encoded rows in an N × 16 KiB buffer; on the first of byte / row / latency cap it forwards the buffer to the supplied flush closure, which the transport layer turns into a chunked-encoding frame.

The struct does not know about HTTP, NDJSON, or chunked transfer — it is wire-agnostic so the gRPC and RedWire paths can reuse it.

Implementations

impl<'a> ChunkProducer<'a>

pub fn new(config: &StreamConfig, clock: &'a dyn Clock) -> Self

pub fn push_line<F>(&mut self, line: &[u8], flush: &mut F) -> Result<bool>

where F: FnMut(&[u8]) -> Result<()>,

Append one already-encoded line (NDJSON: bytes + \n). Returns true if the append triggered a flush.

pub fn drive_lines<S, R, Enc, F>( &mut self, source: S, encode: Enc, flush: &mut F, ) -> Result<u64>

where S: IntoIterator<Item = R>, Enc: FnMut(&R) -> Vec<u8>, F: FnMut(&[u8]) -> Result<()>,

Issue #768 / S9 — drive a pull-based line source into the production buffer. The producer pulls one encoded line at a time from source and routes it through push_line, so the resident working set is the page-aligned buffer plus the single line currently in hand — never the full result set. Pair this with the pull-based scan iterators (parallel_scan::parallel_scan_rows, bitmap_scan::execute_bitmap_scan_stream) whose records are encoded lazily by encode.

Returns the number of lines consumed. Flush caps (byte / row / latency) fire mid-drain exactly as they would for hand-driven push_line calls, so first-line latency stays bounded by chunk.max_latency_ms regardless of how many lines the source will ultimately yield.

pub fn finish<F>(&mut self, flush: &mut F) -> Result<()>

where F: FnMut(&[u8]) -> Result<()>,

Force-flush any buffered bytes — used after the final NDJSON line ({"end": …}) to push the tail of the buffer before closing the connection.

pub fn total_flushes(&self) -> u64

pub fn total_bytes(&self) -> u64

pub fn total_rows(&self) -> u64

pub fn last_flush_reason(&self) -> Option<FlushReason>