Skip to main content

Module array

Module array 

Source
Expand description

The array families and the Array trait that unifies them.

§Why trait objects rather than a closed enum

The task allowed either Arc<dyn Array> or a closed ArrayEnum. AstRS uses the sealed trait plus ArrayRef, for three reasons that all come back to keeping zero-copy slicing simple:

  1. Slicing stays on the concrete type. Every array has an inherent slice(&self, offset, len) -> Self that clones Arcs and adjusts windows — no allocation, no data type re-dispatch. The trait’s Array::slice is a thin Arc::new wrapper over it. With a closed enum, every slice would go through a 23-arm match that reconstructs the variant, and generic code (PrimitiveArray<T>) could not be sliced without first widening it.
  2. Generic families stay generic. PrimitiveArray<T> covers eleven scalar types and GenericStringArray<O> covers two offset widths. An enum would have to enumerate all thirteen instantiations by hand, and downstream code written against PrimitiveArray<i32> would lose its static type on every round trip through the enum.
  3. Arc<dyn Array> is already the sharing unit. A record batch column, a list array’s child and a struct array’s field are all shared, refcounted values; Arc<dyn Array> clones in one atomic increment, while an enum in an Arc adds a layer or forces a large memcpy.

The cost of the trait-object choice is that exhaustiveness is not checked by the compiler: stage 2’s IPC encoder matches on Array::data_type and downcasts. The sealed supertrait ([crate::sealed::Sealed]) keeps the set closed anyway — no downstream crate can add a variant — and array_kinds lists every family so a test can assert full coverage.

Equality across trait objects works too, so ArrayRef and Vec<ArrayRef> compare structurally and crate::RecordBatch derives PartialEq:

use astrs_data::array::{Array, ArrayRef, Int32Array, IntoArrayRef};

let a: ArrayRef = Int32Array::from_values([1, 2, 3]).into_array_ref();
let b: ArrayRef = Int32Array::from_values([1, 2, 3]).into_array_ref();
assert_eq!(a, b);
assert_ne!(a, Int32Array::from_values([1, 2]).into_array_ref());

§The slicing convention

Every slice in this crate — on crate::Buffer, crate::Bitmap, every array and crate::RecordBatch — clamps an out-of-range request to the largest valid window instead of panicking, and every one of them has a try_slice counterpart that reports crate::DataError::SliceOutOfBounds instead. See Array::slice.

Re-exports§

pub use crate::array::binary::BinaryArray;
pub use crate::array::binary::GenericBinaryArray;
pub use crate::array::binary::LargeBinaryArray;
pub use crate::array::boolean::BooleanArray;
pub use crate::array::fixed_size_binary::FixedSizeBinaryArray;
pub use crate::array::fixed_size_list::FixedSizeListArray;
pub use crate::array::iter::ArrayAccessor;
pub use crate::array::iter::ArrayIter;
pub use crate::array::list::ListArray;
pub use crate::array::null::NullArray;
pub use crate::array::offset::OffsetSizeTrait;
pub use crate::array::offset::validate_offsets;
pub use crate::array::primitive::Float16Array;
pub use crate::array::primitive::Float32Array;
pub use crate::array::primitive::Float64Array;
pub use crate::array::primitive::Int8Array;
pub use crate::array::primitive::Int16Array;
pub use crate::array::primitive::Int32Array;
pub use crate::array::primitive::Int64Array;
pub use crate::array::primitive::PrimitiveArray;
pub use crate::array::primitive::UInt8Array;
pub use crate::array::primitive::UInt16Array;
pub use crate::array::primitive::UInt32Array;
pub use crate::array::primitive::UInt64Array;
pub use crate::array::string::GenericStringArray;
pub use crate::array::string::LargeStringArray;
pub use crate::array::string::StringArray;
pub use crate::array::structs::StructArray;
pub use crate::array::temporal::DurationArray;
pub use crate::array::temporal::TimestampArray;

Modules§

binary
GenericBinaryArray — the variable-length byte column, in its Binary (32-bit offsets) and LargeBinary (64-bit offsets) forms.
boolean
BooleanArray — the bit-packed Bool column.
fixed_size_binary
FixedSizeBinaryArray — a column of equally sized byte blobs.
fixed_size_list
FixedSizeListArray — a column whose every value is exactly size child values.
iter
The shared null-aware iterator every array family reuses.
list
ListArray — a column whose every value is a variable-length run of child values.
null
NullArray — the type whose every value is null and whose every buffer is absent.
offset
OffsetSizeTrait — the i32/i64 switch behind the Binary/ LargeBinary and Utf8/LargeUtf8 pairs, plus shared offset validation.
primitive
PrimitiveArray — the fixed-width scalar column, generic over ArrowNativeType.
string
GenericStringArray — the variable-length UTF-8 column, in its Utf8 (32-bit offsets) and LargeUtf8 (64-bit offsets) forms.
structs
StructArray — a column of named child columns, all the same length.
temporal
TimestampArray and DurationArray — the two nanosecond temporal columns.

Traits§

Array
The behaviour every array family provides.
ArrayExt
Downcast helpers on dyn Array.
IntoArrayRef
Wraps a concrete array in an ArrayRef.

Functions§

array_kinds
The name of every array family in the crate, in DataType order.
new_empty_array
An empty array of the given type.
new_null_array
An all-null array of the given type and length.
slice_array
Slices an ArrayRef, short-circuiting a full-range request to an Arc clone instead of rebuilding the array.
try_slice
Checked slicing for any array.

Type Aliases§

ArrayRef
A shared, immutable, dynamically typed column.