Struct StreamingAggregator 

pub struct StreamingAggregator { /* private fields */ }

Constant-memory aggregation state that batches are pushed into.

Aggregation does not need to see its input all at once: every supported function has an incremental form, so the state is O(groups × fields) and each batch can be dropped as soon as it has been folded in. Holding the input instead would make a SELECT avg(x) FROM m over a week of 1 s data cost gigabytes for a single output row.

Rows must carry the group-by and field columns by name; a batch missing a column simply contributes nothing for it, which is how schema evolution across segments is tolerated.

Implementations

impl StreamingAggregator

pub fn new( functions: &[AggFn], field_columns: &[&str], group_by: &[&str], ) -> Self

Create an aggregator for functions over field_columns, grouped by group_by (empty for a whole-dataset aggregate).

pub fn push( &mut self, batch: &RecordBatch, memory_tracker: Option<&MemoryTracker>, ) -> Result<()>

Fold one batch into the running state. The batch may be dropped after.

Errors

Returns an error if the optional memory_tracker budget is exceeded by newly discovered groups.

pub fn finish( self, memory_tracker: Option<&MemoryTracker>, ) -> Result<RecordBatch>

Finalise the accumulated state into a result batch.

Errors

Returns an error if the result batch cannot be built or the optional memory_tracker budget is exceeded.