Struct StructArray

pub struct StructArray { /* private fields */ }

A struct array that stores multiple named fields as columns, similar to a database row.

This mirrors the Apache Arrow Struct array encoding and provides a columnar representation of structured data where each row contains multiple named fields of potentially different types.

Data Layout

The struct array uses a columnar layout where:

• Each field is stored as a separate child array
• All fields must have the same length (number of rows)
• Field names and types are defined in the struct’s dtype
• An optional validity mask indicates which entire rows are null

Row-level nulls

The StructArray contains its own top-level nulls, which are superimposed on top of the field-level validity values. This can be the case even if the fields themselves are non-nullable, accessing a particular row can yield nulls even if all children are valid at that position.

use vortex_array::arrays::{StructArray, BoolArray};
use vortex_array::validity::Validity;
use vortex_array::IntoArray;
use vortex_dtype::FieldNames;
use vortex_buffer::buffer;

// Create struct with all non-null fields but struct-level nulls
let struct_array = StructArray::try_new(
    FieldNames::from(["a", "b", "c"]),
    vec![
        buffer![1i32, 2i32].into_array(),  // non-null field a
        buffer![10i32, 20i32].into_array(), // non-null field b  
        buffer![100i32, 200i32].into_array(), // non-null field c
    ],
    2,
    Validity::Array(BoolArray::from_iter([true, false]).into_array()), // row 1 is null
).unwrap();

// Row 0 is valid - returns a struct scalar with field values
let row0 = struct_array.scalar_at(0).unwrap();
assert!(!row0.is_null());

// Row 1 is null at struct level - returns null even though fields have values
let row1 = struct_array.scalar_at(1).unwrap();
assert!(row1.is_null());

Name uniqueness

It is valid for a StructArray to have multiple child columns that have the same name. In this case, any accessors that use column names will find the first column in sequence with the name.

use vortex_array::arrays::StructArray;
use vortex_array::validity::Validity;
use vortex_array::IntoArray;
use vortex_dtype::FieldNames;
use vortex_buffer::buffer;

// Create struct with duplicate "data" field names
let struct_array = StructArray::try_new(
    FieldNames::from(["data", "data"]),
    vec![
        buffer![1i32, 2i32].into_array(),   // first "data"
        buffer![3i32, 4i32].into_array(),   // second "data"
    ],
    2,
    Validity::NonNullable,
).unwrap();

// field_by_name returns the FIRST "data" field
let first_data = struct_array.field_by_name("data").unwrap();
assert_eq!(first_data.scalar_at(0).unwrap(), 1i32.into());

Field Operations

Struct arrays support efficient column operations:

• Projection: Select/reorder fields without copying data
• Field access: Get columns by name or index
• Column addition: Add new fields to create extended structs
• Column removal: Remove fields to create narrower structs

Validity Semantics

• Row-level nulls are tracked in the struct’s validity child
• Individual field nulls are tracked in each field’s own validity
• A null struct row means all fields in that row are conceptually null
• Field-level nulls can exist independently of struct-level nulls

Examples

use vortex_array::arrays::{StructArray, PrimitiveArray};
use vortex_array::validity::Validity;
use vortex_array::IntoArray;
use vortex_dtype::FieldNames;
use vortex_buffer::buffer;

// Create arrays for each field
let ids = PrimitiveArray::new(buffer![1i32, 2, 3], Validity::NonNullable);
let names = PrimitiveArray::new(buffer![100u64, 200, 300], Validity::NonNullable);

// Create struct array with named fields
let struct_array = StructArray::try_new(
    FieldNames::from(["id", "score"]),
    vec![ids.into_array(), names.into_array()],
    3,
    Validity::NonNullable,
).unwrap();

assert_eq!(struct_array.len(), 3);
assert_eq!(struct_array.names().len(), 2);

// Access field by name
let id_field = struct_array.field_by_name("id").unwrap();
assert_eq!(id_field.len(), 3);

Implementations

impl StructArray

pub fn fields(&self) -> &[ArrayRef]

pub fn field_by_name(&self, name: impl AsRef<str>) -> VortexResult<&ArrayRef>

pub fn field_by_name_opt(&self, name: impl AsRef<str>) -> Option<&ArrayRef>

pub fn names(&self) -> &FieldNames

pub fn struct_fields(&self) -> &StructFields

pub fn new_with_len(len: usize) -> Self

Create a new StructArray with the given length, but without any fields.

pub fn try_new( names: FieldNames, fields: Vec<ArrayRef>, length: usize, validity: Validity, ) -> VortexResult<Self>

pub fn try_new_with_dtype( fields: Vec<ArrayRef>, dtype: StructFields, length: usize, validity: Validity, ) -> VortexResult<Self>

pub fn from_fields<N: AsRef<str>>(items: &[(N, ArrayRef)]) -> VortexResult<Self>

pub fn try_from_iter_with_validity<N: AsRef<str>, A: IntoArray, T: IntoIterator<Item = (N, A)>>( iter: T, validity: Validity, ) -> VortexResult<Self>

pub fn try_from_iter<N: AsRef<str>, A: IntoArray, T: IntoIterator<Item = (N, A)>>( iter: T, ) -> VortexResult<Self>

pub fn project(&self, projection: &[FieldName]) -> VortexResult<Self>

Return a new StructArray with the given projection applied.

Projection does not copy data arrays. Projection is defined by an ordinal array slice which specifies the new ordering of columns in the struct. The projection can be used to perform column re-ordering, deletion, or duplication at a logical level, without any data copying.

pub fn remove_column(&mut self, name: impl Into<FieldName>) -> Option<ArrayRef>

Removes and returns a column from the struct array by name. If the column does not exist, returns None.

pub fn with_column( &self, name: impl Into<FieldName>, array: ArrayRef, ) -> VortexResult<Self>

Create a new StructArray by appending a new column onto the existing array.

impl StructArray

pub fn into_record_batch(self) -> VortexResult<RecordBatch>

pub fn into_record_batch_with_schema( self, schema: &Schema, ) -> VortexResult<RecordBatch>

Methods from Deref<Target = dyn Array>

pub fn display_values(&self) -> impl Display

Display logical values of the array

For example, an i16 typed array containing the first five non-negative integers is displayed as: [0i16, 1i16, 2i16, 3i16, 4i16].

Examples

let array = buffer![0_i16, 1, 2, 3, 4].into_array();
assert_eq!(
    format!("{}", array.display_values()),
    "[0i16, 1i16, 2i16, 3i16, 4i16]",
)

See also: Array::display_as, DisplayArrayAs, and DisplayOptions.

pub fn display_as(&self, options: DisplayOptions) -> impl Display

Display the array as specified by the options.

See DisplayOptions for examples.

pub fn display_tree(&self) -> impl Display

Display the tree of encodings of this array as an indented lists.

While some metadata (such as length, bytes and validity-rate) are included, the logical values of the array are not displayed. To view the logical values see Array::display_as and DisplayOptions.

Examples

let array = buffer![0_i16, 1, 2, 3, 4].into_array();
let expected = "root: vortex.primitive(i16, len=5) nbytes=10 B (100.00%)
  metadata: EmptyMetadata
  buffer (align=2): 10 B (100.00%)
";
assert_eq!(format!("{}", array.display_tree()), expected);

pub fn as_<V: VTable>(&self) -> &V::Array

Returns the array downcast to the given A.

pub fn as_opt<V: VTable>(&self) -> Option<&V::Array>

Returns the array downcast to the given A.

pub fn is<V: VTable>(&self) -> bool

Is self an array with encoding from vtable V.

pub fn is_constant(&self) -> bool

pub fn is_constant_opts(&self, cost: Cost) -> bool

pub fn as_constant(&self) -> Option<Scalar>

pub fn nbytes(&self) -> u64

Total size of the array in bytes, including all children and buffers.

pub fn to_array_iterator(&self) -> impl ArrayIterator + 'static

Create an ArrayIterator over the array.

pub fn serialize( &self, ctx: &ArrayContext, options: &SerializeOptions, ) -> VortexResult<Vec<ByteBuffer>>

Serialize the array into a sequence of byte buffers that should be written contiguously. This function returns a vec to avoid copying data buffers.

Optionally, padding can be included to guarantee buffer alignment and ensure zero-copy reads within the context of an external file or stream. In this case, the alignment of the first byte buffer should be respected when writing the buffers to the stream or file.

The format of this blob is a sequence of data buffers, possible with prefixed padding, followed by a flatbuffer containing an fba::Array message, and ending with a little-endian u32 describing the length of the flatbuffer message.

pub fn to_array_stream(&self) -> impl ArrayStream + 'static

Create an ArrayStream over the array.

pub fn as_null_typed(&self) -> NullTyped<'_>

Downcasts the array for null-specific behavior.

pub fn as_bool_typed(&self) -> BoolTyped<'_>

Downcasts the array for bool-specific behavior.

pub fn as_primitive_typed(&self) -> PrimitiveTyped<'_>

Downcasts the array for primitive-specific behavior.

pub fn as_decimal_typed(&self) -> DecimalTyped<'_>

Downcasts the array for decimal-specific behavior.

pub fn as_utf8_typed(&self) -> Utf8Typed<'_>

Downcasts the array for utf8-specific behavior.

pub fn as_binary_typed(&self) -> BinaryTyped<'_>

Downcasts the array for binary-specific behavior.

pub fn as_struct_typed(&self) -> StructTyped<'_>

Downcasts the array for struct-specific behavior.

pub fn as_list_typed(&self) -> ListTyped<'_>

Downcasts the array for list-specific behavior.

pub fn as_extension_typed(&self) -> ExtensionTyped<'_>

Downcasts the array for extension-specific behavior.

Trait Implementations

impl AsRef<dyn Array> for StructArray

fn as_ref(&self) -> &dyn Array

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for StructArray

fn clone(&self) -> StructArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StructArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Deref for StructArray

type Target = dyn Array

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl From<StructArray> for ArrayRef

fn from(value: StructArray) -> ArrayRef

Converts to this type from the input type.

impl IntoArray for StructArray

fn into_array(self) -> ArrayRef

impl ValidityHelper for StructArray

fn validity(&self) -> &Validity