Struct SuperTable 

pub struct SuperTable {
    pub batches: Vec<Arc<Table>>,
    pub schema: Vec<Arc<Field>>,
    pub n_rows: usize,
    pub name: String,
}

SuperTable

Higher-order container representing a sequence of Table batches with consistent schema.

Overview

• Each batch is a Table (record batch) with identical column metadata.
• Stored as Vec<Arc<Table>>, preserving order and schema consistency.
• Row counts per batch may vary, but are consistent across all Table columns.
• When exported via Arrow FFI, the batches are viewed as a single logical table.
• Useful for open-ended streams, partitioned datasets, or other scenarios where batches are processed independently.

Fields

• batches: ordered collection of Table batches.
• schema: cached schema from the first batch for fast access.
• n_rows: total row count across all batches.
• name: super table name.

Use cases

• Streaming and mini-batch processing.
• Reading multiple Arrow IPC/memory-mapped files as one dataset.
• Parallel or windowed in-memory analytics.
• Incremental table construction where batches arrive over time.

Fields

batches: Vec<Arc<Table>>

schema: Vec<Arc<Field>>

n_rows: usize

name: String

Implementations

impl SuperTable

pub fn new(name: String) -> Self

Creates a new empty BatchedTable with a specified name.

pub fn from_batches( batches: Vec<Arc<Table>>, name_override: Option<String>, ) -> Self

Builds from a collection of Table batches.

Panics if column count or field metadata are inconsistent.

pub fn push(&mut self, batch: Arc<Table>)

Append a new Table batch.

Panics on schema mismatch.

pub fn to_table(&self, name: Option<&str>) -> Table

Materialise a single Table containing all rows (concatenated).

Uses arc data clones

pub fn n_cols(&self) -> usize

pub fn n_batches(&self) -> usize

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn schema(&self) -> &[Arc<Field>]

pub fn batches(&self) -> &[Arc<Table>]

pub fn batch(&self, idx: usize) -> Option<&Arc<Table>>

pub fn view(&self, offset: usize, len: usize) -> SuperTableV

pub fn from_views(slices: &[TableV], name: String) -> Self

Trait Implementations

impl Clone for SuperTable

fn clone(&self) -> SuperTable

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SuperTable

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SuperTable

fn default() -> Self

Returns the “default value” for a type.

impl Display for SuperTable

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FromIterator<Table> for SuperTable

fn from_iter<T: IntoIterator<Item = Table>>(iter: T) -> Self

Creates a value from an iterator.

impl PartialEq for SuperTable

fn eq(&self, other: &SuperTable) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for SuperTable