Module chunk 

DataChunk: Basic unit of streaming execution

A DataChunk represents a fixed-size batch of rows processed in streaming mode. Typical size: 2048 rows (~8MB, OLAP vectorized batch)

Ownership & Memory Accounting Rules

• rows: Owned Vec<Vec<Value>>. Clone performs an explicit deep copy (there is no shallow-clone path).
• schema: Arc<Schema> — shared reference, cheap Arc::clone on Clone.
• layout: Arc<SlotLayout> — always present; shared reference, cheap on Clone.
• memory_reservation: Option<MemoryReservation> / Option<MemoryPoolReservation> — ownership stays with the original chunk on Clone; the clone gets None. Use take_memory_reservation() to transfer ownership explicitly.

Move-first discipline

New code moves rows instead of cloning them:

• expand_visible_rows(): the single terminal move (visible rows out, expanded by multiplicity); used by LocalChunkCollector and every collect path.
• visible_rows(): the single borrow of visible rows (selection aware).
• take_indices/slice: move a subset of rows into a new chunk (efficient, uses std::mem::take per row), preserving multiplicity.
• Clone: the single explicit deep copy. It drops memory reservations and the deferred-build flag; the new owner must re-account memory via its own pool/tracker.

Construction Paths

• new_with_layout(rows, layout) — Production path. Schema is derived from layout slot metadata. Always preferred when a SlotLayout is available.
• new(rows, schema) — Schema-driven path. Layout auto-created from column names.
• from_rows(rows) / from_rows_with_col_names(rows, col_names) — Convenience constructors for tests. Always produce a layout (auto-created).
• project_columns(columns, layout) — Designated Project output path; rebuilds the typed layout unconditionally because the output is chunk-bounded. from_columns is the crate-private row-major fixture behind it.

Columnar gating

Fresh column sets from storage scans and join outputs honor use_columnar_path; small 1:1 rebuilds (project, assign) rebuild unconditionally by design. Blocking accumulation uses ColumnarBatch (typed) for sort/topn and MaterializedBatch (spill form) elsewhere.

Structs

ColumnInfo

ColumnarBatch

Column-major accumulation of a full relation (one BatchColumn per output column), used by blocking operators below the spill boundary.

ColumnarPolicy

Adaptive gate for the typed columnar chunk layout (cross-query shared).

DataChunk

A chunk of rows processed in streaming execution

EdgeHeaderColumn

Struct-of-arrays edge identity column: one (src, dst, edge_type, ranking) tuple per row with no property map and no storage-internal identifier. All four vectors always share one length; every constructor and mutator below maintains that invariant together, so element access stays index-aligned without per-access checks.

LocalChunkCollector

Thread-local terminal accumulator for query results.

QueryColumnarOverride

Per-query override for the shared ColumnarPolicy decision.

RowBufferPool

Process-global row-buffer pool.

Schema

Simple schema representation

Enums

BatchColumn

TypedColumn

TypedKind

Kind of a typed fixed-size scalar column.

Constants

COLLECTOR_BLOCK_ROWS

Rows per collector block. Bounds the reallocation peak of a single push.

DEFAULT_CHUNK_SIZE

Alias for the default chunk size (kept in sync with ExecutionContext::DEFAULT_CHUNK_SIZE).

MAX_POOLED_ROWS

MAX_POOLED_ROW_CAPACITY

VECTORIZED_BATCH_SIZE

OLAP vectorized batch size.