Type Alias ChunkedTable

Source

pub type ChunkedTable = SuperTable;

Expand description

§ChunkedTable

Batched (windowed/chunked) table - collection of Tables.

§Data structure

Each Table represents a record batch with schema consistency enforced. Windows/row-chunks are tracked as Vec

.

batches: Ordered record batches.
schema: schema of the first batch, cached for fast access.
n_rows: total number of rows across all batches.
name: logical group name.

§Use cases

Useful for cases such as : 1. Streaming (and mini-batch processing) 2. Reading from multiple memory mapped arrow files from disk 3. In-memory analytics, where chunks can be used as a source for parallelism, or windowing, etc., depending on use case semantics.

Aliased Type§

pub struct ChunkedTable {
    pub batches: Vec<Arc<Table>>,
    pub schema: Vec<Arc<Field>>,
    pub n_rows: usize,
    pub name: String,
}

Fields§

§batches: Vec<Arc<Table>>§schema: Vec<Arc<Field>>§n_rows: usize§name: String

ChunkedTable