Struct Table 

#[repr(C, align(64))]
pub struct Table {
    pub cols: Vec<FieldArray>,
    pub n_rows: usize,
    pub name: String,
    pub metadata: BTreeMap<String, String>,
}

Table

Description

• Standard columnar table with named columns (FieldArray), a fixed number of rows, and an optional logical table name.
• All columns are required to be equal length and have consistent schema.
• Supports zero-copy slicing, efficient iteration, and bulk operations.
• Equivalent to the RecordBatch in Apache Arrow.

Structure

• cols: A vector of FieldArray, each representing a column with metadata and data.
• n_rows: The logical number of rows (guaranteed equal for all columns).
• name: Optional logical name or alias for this table instance.

Usage

• Use Table as a general-purpose, in-memory columnar data container.
• Good for analytics, and transformation pipelines.
• For batched/partitioned tables, see SuperTable or windowed/chunked abstractions.
• Cast into Polars dataframe via .to_polars() or Apache Arrow via .to_apache_arrow()
• FFI-compatible

Notes

• Table instances are typically lightweight to clone and pass by value.
• For mutation, construct a new table or replace individual columns as needed.
• There is an alias RecordBatch under crate::aliases::RecordBatch

Example

use minarrow::{fa_i32, fa_str32, Print, Table};

let col1 = fa_i32!("numbers", 1, 2, 3);
let col2 = fa_str32!("letters", "x", "y", "z");

let mut tbl = Table::new("Demo".into(), vec![col1, col2].into());
tbl.print();

Fields

cols: Vec<FieldArray>

FieldArrays representing named columns.

n_rows: usize

Number of rows in the table.

name: String

Table name

metadata: BTreeMap<String, String>

Schema-level metadata as key-value pairs. Captures metadata that Arrow producers like PyArrow embed in the top-level ArrowSchema.metadata, e.g. pandas categorical ordering.

Implementations

impl Table

pub fn new(name: String, cols: Option<Vec<FieldArray>>) -> Self

Constructs a new Table with a specified name and optional columns. If cols is provided, the number of rows will be inferred from the first column.

pub fn new_with_metadata( name: String, cols: Option<Vec<FieldArray>>, metadata: BTreeMap<String, String>, ) -> Self

Constructs a new Table with schema-level metadata.

Use this when importing data from Arrow producers that embed metadata in the top-level schema, e.g. pandas categorical ordering via PyArrow.

pub fn metadata(&self) -> &BTreeMap<String, String>

Returns a reference to the schema-level metadata.

pub fn new_empty() -> Self

Constructs a new, empty Table with a globally unique name.

pub fn from_arena( name: String, schema: &[Arc<Field>], arena: Arena, regions: Vec<AAMaker>, n_rows: usize, ) -> Self

Build a Table from an Arena and its collected array regions.

Freezes the arena into a SharedBuffer, then reconstructs each column as a zero-copy view into the shared allocation. This is the read-side complement to Arena::write_slices and friends.

Typical use: IPC or streaming ingestion where batch sizes are known from message headers, allowing a single arena allocation per batch.

pub fn add_col(&mut self, field_array: FieldArray)

Adds a column with a name.

pub fn schema(&self) -> Vec<Arc<Field>>

Builds a schema via the underlying field arrays

pub fn n_cols(&self) -> usize

Returns the number of columns.

pub fn n_rows(&self) -> usize

Returns the number of rows.

pub fn is_empty(&self) -> bool

Returns true if the table is empty (no columns or no rows).

pub fn col_names(&self) -> Vec<&str>

Returns the list of column names.

pub fn rename_columns( &mut self, mapping: &[(&str, &str)], ) -> Result<(), MinarrowError>

Rename columns in place. Each pair is (old_name, new_name).

Returns an error if any old name is not found. This is metadata-only - array data is not touched.

pub fn col_name_index(&self, name: &str) -> Option<usize>

Returns the index of a column by name.

pub fn col_numeric(&self, name: &str) -> Result<NumericArrayV, MinarrowError>

Resolve a named column to a NumericArrayV.

pub fn col_text(&self, name: &str) -> Result<TextArrayV, MinarrowError>

Resolve a named column to a TextArrayV.

pub fn col_bitmask(&self, name: &str) -> Result<BitmaskV, MinarrowError>

Resolve a named column to a BitmaskV.

pub fn delete_range(&mut self, start: usize, end: usize)

Removes the rows in [start, end) from every column, shifting later rows left.

Columns delete in place through Vec64::delete_range. Shared columns and shared buffers are cloned first i.e. copy-on-write.

Panics

Panics if start > end or end > n_rows.

pub fn remove_col(&mut self, name: &str) -> bool

Removes a column by name.

pub fn remove_col_at(&mut self, idx: usize) -> bool

Removes a column by index.

pub fn clear(&mut self)

Clears all columns and resets row count.

pub fn has_col(&self, name: &str) -> bool

Checks if a column with the given name exists.

pub fn cols(&self) -> &[FieldArray]

Returns all columns as a slice.

pub fn cols_mut(&mut self) -> &mut [FieldArray]

Returns mutable reference to all columns.

pub fn iter(&self) -> Iter<'_, FieldArray>

pub fn iter_mut(&mut self) -> IterMut<'_, FieldArray>

pub fn set_name(&mut self, name: impl Into<String>)

pub fn len(&self) -> usize

pub fn slice_clone(&self, offset: usize, len: usize) -> Self

Returns a new owned Table containing rows [offset, offset+len).

All columns are deeply copied, but only for the affected row(s).

pub fn slice(&self, offset: usize, len: usize) -> TableV

Returns a zero-copy view over rows [offset, offset+len). This view borrows from the parent table and does not copy data.

pub fn map_col<T, F>(&self, col_name: &str, func: F) -> Option<T>

where F: FnOnce(&FieldArray) -> T,

Maps a function over a single column by name, returning the result. Returns None if the column doesn’t exist.

pub fn map_cols_by_name<T, F>(&self, col_names: &[&str], func: F) -> Vec<T>

where F: FnMut(&FieldArray) -> T,

Maps a function over multiple columns by name, returning a Vec of results. Warns if any requested columns are missing.

pub fn map_cols_by_index<T, F>(&self, indices: &[usize], func: F) -> Vec<T>

where F: FnMut(&FieldArray) -> T,

Maps a function over multiple columns by index, returning a Vec of results. Warns if any requested indices are out of bounds.

pub fn map_all_cols<T, F>(&self, func: F) -> Vec<T>

where F: FnMut(&FieldArray) -> T,

Maps a function over all columns, returning a Vec of results.

pub fn apply_cols<E>( &self, f: impl FnMut(&FieldArray) -> Result<FieldArray, E>, ) -> Result<Table, E>

Apply a transformation to each column, producing a new table.

The closure receives each FieldArray and returns a transformed FieldArray. To pass a column through unchanged, clone it. The closure can dispatch on Array variant to handle numeric, text, temporal, and boolean columns differently.

pub fn insert_rows( &mut self, index: usize, other: &Self, ) -> Result<(), MinarrowError>

Inserts rows from another table at the specified index.

This is an O(n) operation where n is the number of rows after the insertion point.

Arguments

• index - Position before which to insert (0 = prepend, n_rows = append)
• other - Table to insert

Requirements

• Both tables must have the same number of columns
• Column names, types, and nullability must match in order
• index must be <= self.n_rows()

Errors

• IndexError if index > n_rows
• IncompatibleTypeError if column schemas don’t match

pub fn split(self, index: usize) -> Result<SuperTable, MinarrowError>

Splits the Table at the specified row index, consuming self and returning a SuperTable with two Table batches.

Splits the underlying buffers, allocating new storage for the second half.

impl Table

pub fn par_iter(&self) -> Iter<'_, FieldArray>

pub fn par_iter_mut(&mut self) -> IterMut<'_, FieldArray>

pub fn to_apache_arrow(&self) -> RecordBatch

Export each column to arrow-rs ArrayRef and build a RecordBatch.

The Arrow schema is derived from the imported array dtypes while preserving the original field names and nullability flags.

Panics on FFI failure or empty table. For a fallible variant, see Table::try_to_apache_arrow.

pub fn try_to_apache_arrow(&self) -> Result<RecordBatch, MinarrowError>

Fallible variant of Table::to_apache_arrow.

pub fn to_polars(&self) -> DataFrame

Casts the table to a Polars DataFrame.

Panics on FFI failure. For a fallible variant, see Table::try_to_polars.

pub fn try_to_polars(&self) -> Result<DataFrame, MinarrowError>

Fallible variant of Table::to_polars.

pub fn from_apache_arrow(rb: &RecordBatch) -> Table

Build a Table from an arrow-rs RecordBatch. Column names, dtypes, nullability, and any custom field metadata are recovered from the RecordBatch schema. The Table name will be empty; assign one via Table::new if needed.

Panics on FFI failure. For a fallible variant, see Table::try_from_apache_arrow.

pub fn try_from_apache_arrow(rb: &RecordBatch) -> Result<Table, MinarrowError>

Fallible variant of Table::from_apache_arrow.

pub fn from_polars(df: &DataFrame) -> Table

Build a Table from a Polars DataFrame.

A polars DataFrame has columns that are inherently multi-chunked; the canonical mapping is DataFrame <-> crate::SuperTable. This helper routes through crate::SuperTable::from_polars and then consolidates the batches into a single contiguous Table with 64-byte aligned column buffers.

Column names, dtypes, nullability, and any custom Series metadata are recovered.

Performance note

Two separate costs to be aware of:

1. Alignment copy: Polars data is typically 8-byte aligned (per the Arrow spec default), while Minarrow uses 64-byte aligned Vec64<T> buffers for SIMD. Most of the time this results in a memory copy to realign on import, unless the source data happens to be pre-aligned to 64 bytes. The FFI hand-off itself is pointer-level zero-copy; the realignment is done by Buffer::from_shared when the source isn’t 64-byte aligned.

2. Consolidation copy: Multi-chunk columns are merged into a single contiguous buffer per column, which is a second O(n) allocation and copy pass. Single-chunk DataFrames (e.g. after df.align_chunks_par() then df.rechunk() on the caller side) skip this step. The consolidation itself is cheap on Linux when the vmap64 feature is enabled.

In practice you should expect at least one full allocation + copy when importing a polars DataFrame into a Table. If you would like to preserve the original chunk boundaries and avoid the consolidation step, use crate::SuperTable::from_polars directly - though the alignment copy will still occur per chunk that isn’t pre-aligned.

Panics on FFI failure. For a fallible variant, see Table::try_from_polars.

pub fn try_from_polars(df: &DataFrame) -> Result<Table, MinarrowError>

Fallible variant of Table::from_polars.

Trait Implementations

impl Add for Table

type Output = Result<Table, MinarrowError>

The resulting type after applying the + operator.

fn add(self, rhs: Self) -> Self::Output

Performs the + operation.

impl ByteSize for Table

ByteSize for Table - sum of all column arrays

fn est_bytes(&self) -> usize

Returns the estimated byte size of this object in memory.

impl Clone for Table

fn clone(&self) -> Table

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ColumnSelection for Table

Available on crate features select and views only.

type View = TableV

The view type returned by multi-field selection e.g. TableV

type ColumnView = ArrayV

A single column view by index e.g. ArrayV

type ColumnOwned = FieldArray

An owned single field by name via .get() e.g. FieldArray for Table, Arc for Cube

fn c<S: FieldSelector>(&self, selection: S) -> TableV

Select fields/columns by name, index, or range

fn get(&self, field: &str) -> Option<FieldArray>

Get a single field by name, returning an owned value.

fn col_ix(&self, idx: usize) -> Option<ArrayV>

Get a single column view by index

fn col_vec(&self) -> Vec<ArrayV>

Get all columns as views

fn get_cols(&self) -> Vec<Arc<Field>>

Get the fields for field resolution

fn col(&self, name: &str) -> Self::View

Alias for c - select column by name

impl Combine for Table

type Element = FieldArray

Element type the container is composed of.

type Output = Table

Owned container shape produced by combine.

fn combine( &self, elements: Vec<Self::Element>, ) -> Result<Self::Output, MinarrowError>

Build the output from elements, inheriting self’s name, metadata, and any grouping state.

impl Concatenate for Table

fn concat(self, other: Self) -> Result<Self, MinarrowError>

Concatenates two tables vertically (row-wise).

Requirements

• Both tables must have the same number of columns
• Column names, types, and nullability must match in order

Returns

A new Table with rows from self followed by rows from other

Errors

• IncompatibleTypeError if column schemas don’t match

impl Debug for Table

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Table

fn default() -> Table

Returns the “default value” for a type.

impl Display for Table

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Div for Table

type Output = Result<Table, MinarrowError>

The resulting type after applying the / operator.

fn div(self, rhs: Self) -> Self::Output

Performs the / operation.

impl From<&DataFrame> for Table

Available on crate feature cast_polars only.

fn from(df: &DataFrame) -> Self

Converts to this type from the input type.

impl From<&RecordBatch> for Table

Available on crate feature cast_arrow only.

fn from(rb: &RecordBatch) -> Self

Converts to this type from the input type.

impl From<Table> for Value

fn from(v: Table) -> Self

Converts to this type from the input type.

impl From<Table> for TableV

Table -> TableV conversion

fn from(table: Table) -> Self

Converts to this type from the input type.

impl From<TableV> for Table

TableV -> Table conversion. Respects active column selection.

fn from(view: TableV) -> Self

Converts to this type from the input type.

impl FromIterator<Table> for SuperTable

fn from_iter<T: IntoIterator<Item = Table>>(iter: T) -> Self

Creates a value from an iterator.

impl<'a> IntoIterator for &'a Table

type Item = &'a FieldArray

The type of the elements being iterated over.

type IntoIter = Iter<'a, FieldArray>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl<'a> IntoIterator for &'a mut Table

type Item = &'a mut FieldArray

The type of the elements being iterated over.

type IntoIter = IterMut<'a, FieldArray>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl IntoIterator for Table

type Item = FieldArray

The type of the elements being iterated over.

type IntoIter = <Vec<FieldArray> as IntoIterator>::IntoIter

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl Mul for Table

type Output = Result<Table, MinarrowError>

The resulting type after applying the * operator.

fn mul(self, rhs: Self) -> Self::Output

Performs the * operation.

impl PartialEq for Table

fn eq(&self, other: &Table) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Rem for Table

type Output = Result<Table, MinarrowError>

The resulting type after applying the % operator.

fn rem(self, rhs: Self) -> Self::Output

Performs the % operation.

impl RowSelection for Table

Available on crate features select and views only.

type View = TableV

The view type returned by selection operations

fn r<S: DataSelector>(&self, selection: S) -> TableV

Select rows by index or range

fn get_row_count(&self) -> usize

Get the count for data resolution

impl Shape for Table

fn shape(&self) -> ShapeDim

Returns arbitrary Shape dimension for any data shape

fn shape_1d(&self) -> usize

Returns the first dimension shape

fn shape_2d(&self) -> (usize, usize)

Returns the first and second dimension shapes

fn shape_3d(&self) -> (usize, usize, usize)

Returns the first, second and third dimension shapes

fn shape_4d(&self) -> (usize, usize, usize, usize)

Returns the first, second, third and fourth dimension shapes

impl StructuralPartialEq for Table

impl Sub for Table

type Output = Result<Table, MinarrowError>

The resulting type after applying the - operator.

fn sub(self, rhs: Self) -> Self::Output

Performs the - operation.

impl TryFrom<&Table> for Matrix

type Error = MinarrowError

The type returned in the event of a conversion error.

fn try_from(table: &Table) -> Result<Self, Self::Error>

Performs the conversion.

impl TryFrom<Table> for Matrix

type Error = MinarrowError

The type returned in the event of a conversion error.

fn try_from(table: Table) -> Result<Self, Self::Error>

Performs the conversion.

impl TryFrom<Value> for Table

type Error = MinarrowError

The type returned in the event of a conversion error.

fn try_from(v: Value) -> Result<Self, Self::Error>

Performs the conversion.