Crate datafusion_row

This module contains code to translate arrays back and forth to a row based format. The row based format is backed by raw bytes ([[u8]]) and used to optimize certain operations.

In general, DataFusion is a so called “vectorized” execution model, specifically it uses the optimized calculation kernels in arrow to amortize dispatch overhead.

However, as mentioned in this paper, there are some “row oriented” operations in a database that are not typically amenable to vectorization. The “classics” are: hash table updates in joins and hash aggregates, as well as comparing tuples in sort / merging.

Re-exports

pub use layout::row_supported;

pub use layout::RowType;

Modules

accessor

RowAccessor provides a Read/Write/Modify access for row with all fixed-sized fields:

layout

Various row layouts for different use case

reader

read_as_batch converts raw bytes to [RecordBatch]

writer

RowWriter writes [RecordBatch]es to Vec to stitch attributes together

Macros

fn_add_idx

fn_get_idx

fn_get_idx_opt

fn_set_idx

get_idx

set_idx

Structs

MutableRecordBatch

Columnar Batch buffer that assists creating RecordBatches