Struct RecordBatchMemoryCounter 

pub struct RecordBatchMemoryCounter { /* private fields */ }

Tracks the memory used by a sequence of RecordBatches that may share underlying buffers, counting each buffer exactly once.

Use this instead of get_record_batch_memory_size to account for the total memory of a sequence of batches, e.g. when buffering the batches of an input stream. Such batches can share buffers (for example, operators like aggregates emit one large batch as multiple zero-copy slices), and calling get_record_batch_memory_size per batch counts the shared buffers once per batch, while this counter counts them exactly once. A batch’s buffers are kept alive by the batch even when only a sub-range is referenced, so counting unique buffers in full reflects the memory the batches actually retain.

Implementations

impl RecordBatchMemoryCounter

pub fn new() -> Self

pub fn count_batch(&mut self, batch: &RecordBatch) -> usize

Count batch, returning the memory used by its buffers that have not been counted before.

pub fn memory_usage(&self) -> usize

Total memory of the unique buffers of all batches counted so far.

Trait Implementations

impl Debug for RecordBatchMemoryCounter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RecordBatchMemoryCounter

fn default() -> RecordBatchMemoryCounter

Returns the “default value” for a type.