Struct ObjectAccumulator 

pub struct ObjectAccumulator { /* private fields */ }

Accumulates encoded rows for one open object, and says when to roll.

The sink pushes each page’s rows in, takes whatever completed objects come back, and calls finish at flush time for the partial remainder.

Implementations

impl ObjectAccumulator

pub fn new(max_rows: Option<usize>, max_bytes: Option<usize>) -> Self

Build an accumulator. max_rows/max_bytes of None or 0 mean “no limit on this axis”; with neither set, nothing ever rolls and the whole run lands in one object at finish.

pub fn with_part_size(self, bytes: usize) -> Self

Emit Emit::Parts once the unflushed tail reaches bytes, so the sink can stream them into a multipart upload and free the memory.

Without this, an object with a large (or absent) byte cap is held whole in RAM before its single-shot upload — which is what caps output size by available memory. 0 disables parting.

pub fn parted_bytes(&self) -> usize

Bytes of the open object already uploaded as parts.

pub fn has_parts(&self) -> bool

Whether any part of the open object has already been uploaded — the sink needs this to know whether to complete a multipart upload or do a single-shot put.

pub fn rows(&self) -> usize

Rows currently held in the open object.

pub fn len(&self) -> usize

Bytes currently held in the open object.

pub fn is_empty(&self) -> bool

Whether nothing is buffered.

pub fn push_encoded(&mut self, encoded: &[u8]) -> Emit

Append one encoded record (caller-encoded, so the accumulator stays format-agnostic) and return any object this completed.

The threshold is checked after appending, so a single record larger than max_bytes still lands in its own object rather than being refused or split — splitting would corrupt it, and refusing would drop data the source really produced.

pub fn push_record(&mut self, record: &Value) -> Result<Emit, FaucetError>

Append one JSON record as an NDJSON line.

pub fn finish(&mut self) -> Option<CompletedObject>

Close the open object, if any. Called at flush, and on drop-time finalisation — an object left unfinished is data loss, so a sink must never skip it.

Returns Some whenever the object holds rows, including when the unflushed tail is empty but parts were already uploaded: a multipart upload still has to be completed.

Trait Implementations

impl Debug for ObjectAccumulator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.