Struct LocalChunkCollector 

pub struct LocalChunkCollector { /* private fields */ }

Thread-local terminal accumulator for query results.

Implementations

impl LocalChunkCollector

pub fn new(col_names: Vec<String>) -> Self

pub fn attach_spill_manager(&mut self, manager: Arc<SpillManager>)

Attach a spill manager for spill-through collect.

When no explicit threshold was configured via Self::set_spill_threshold_rows, the manager’s effective threshold (config value or default) is adopted. A threshold of u64::MAX (from Some(0) config) disables collector spill.

pub fn push_chunk(&mut self, chunk: &mut DataChunk) -> Result<(), QueryError>

Move all visible (expanded) rows of chunk into the collector, splitting them across fixed-size blocks or spilling them to disk once the threshold is exceeded.

pub fn merge(&mut self, other: LocalChunkCollector) -> Result<(), QueryError>

Merge another collector by moving whole blocks (no per-Value clone).

Spilled runs are concatenated in order; both open writers are finalized first so ordering is preserved. Merging two spilled collectors requires identical column names and the same spill manager (runs live in the manager’s directory); otherwise an error is returned and the caller should drain via into_rows first. Pure in-memory merges keep the previous unchecked behavior.

pub fn total_logical_rows(&self) -> u128

Expanded row count observed (selection + multiplicity applied).

pub fn len(&self) -> usize

Remaining rows (memory + spilled, not yet drained).

pub fn is_empty(&self) -> bool

pub fn col_names(&self) -> &[String]

pub fn finish_spill(&mut self) -> Result<(), QueryError>

Finalize the open run, if any, so all spilled rows are readable and counters are exact. Idempotent; into_rows and drain_rows call it automatically, while streaming collect paths call it explicitly before snapshotting observability counters.

pub fn into_rows(self) -> Result<(Vec<Vec<Value>>, Vec<String>), QueryError>

Full ordered results: spilled runs replayed in write order, then memory blocks. Peak memory is bounded by the largest single run.

pub fn drain_rows(&mut self) -> Result<Vec<Vec<Value>>, QueryError>

Segmented results: each call returns at most one run’s rows, so streaming consumers never hold more than a single run in memory. After the runs are exhausted the memory blocks are returned whole, then empty vectors signal completion.

pub fn into_blocks(self) -> (Vec<Vec<Vec<Value>>>, Vec<String>)

Block-partitioned memory rows.

Only valid for in-memory collectors; spilled state must be consumed via Self::into_rows or Self::drain_rows.

pub fn iter_rows(&self) -> impl Iterator<Item = &Vec<Value>>

Ordered cursor over memory blocks only.

When spilled, the disk portion is NOT included; paged consumers must use Self::drain_rows, full consumers Self::into_rows.

pub fn num_blocks(&self) -> usize

pub fn is_spilled(&self) -> bool

Whether any row has been spilled to disk.

pub fn spilled_rows(&self) -> u64

Rows written to disk so far.

pub fn spilled_bytes(&self) -> u64

Bytes written to disk so far (on-disk file sizes).

pub fn spilled_run_count(&self) -> usize

Number of finalized spill runs.

pub fn set_spill_threshold_rows(&mut self, threshold: u64)

Configure the logical-row threshold that will trigger spill-through collect once a spill manager is attached.

pub fn spill_threshold_exceeded(&self) -> bool

Whether the configured spill threshold has been exceeded.

Trait Implementations

impl Debug for LocalChunkCollector

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LocalChunkCollector

fn default() -> Self

Returns the “default value” for a type.