Struct arrow_data::ArrayData

pub struct ArrayData { /* private fields */ }

An generic representation of Arrow array data which encapsulates common attributes and operations for Arrow array. Specific operations for different arrays types (e.g., primitive, list, struct) are implemented in Array.

Implementations

impl ArrayData

pub unsafe fn new_unchecked(
    data_type: DataType,
    len: usize,
    null_count: Option<usize>,
    null_bit_buffer: Option<Buffer>,
    offset: usize,
    buffers: Vec<Buffer>,
    child_data: Vec<ArrayData>
) -> Self

Create a new ArrayData instance;

If null_count is not specified, the number of nulls in null_bit_buffer is calculated.

If the number of nulls is 0 then the null_bit_buffer is set to None.

Safety

The input values must form a valid Arrow array for data_type, or undefined behavior can results.

Note: This is a low level API and most users of the arrow crate should create arrays using the methods in the array module.

pub fn try_new(
    data_type: DataType,
    len: usize,
    null_bit_buffer: Option<Buffer>,
    offset: usize,
    buffers: Vec<Buffer>,
    child_data: Vec<ArrayData>
) -> Result<Self, ArrowError>

Create a new ArrayData, validating that the provided buffers form a valid Arrow array of the specified data type.

If the number of nulls in null_bit_buffer is 0 then the null_bit_buffer is set to None.

Note: This is a low level API and most users of the arrow crate should create arrays using the methods in the array module.

pub const fn builder(data_type: DataType) -> ArrayDataBuilder

Returns a builder to construct a ArrayData instance.

pub const fn data_type(&self) -> &DataType

Returns a reference to the data type of this array data

pub fn buffers(&self) -> &[Buffer]

Returns a slice of buffers for this array data

pub fn child_data(&self) -> &[ArrayData]

Returns a slice of children data arrays

pub fn is_null(&self, i: usize) -> bool

Returns whether the element at index i is null

pub const fn null_bitmap(&self) -> Option<&Bitmap>

Returns a reference to the null bitmap of this array data

pub fn null_buffer(&self) -> Option<&Buffer>

Returns a reference to the null buffer of this array data.

pub fn is_valid(&self, i: usize) -> bool

Returns whether the element at index i is not null

pub const fn len(&self) -> usize

Returns the length (i.e., number of elements) of this array

pub const fn is_empty(&self) -> bool

pub const fn offset(&self) -> usize

Returns the offset of this array

pub const fn null_count(&self) -> usize

Returns the total number of nulls in this array

pub fn get_buffer_memory_size(&self) -> usize

Returns the total number of bytes of memory occupied by the buffers owned by this ArrayData.

pub fn get_array_memory_size(&self) -> usize

Returns the total number of bytes of memory occupied physically by this ArrayData.

pub fn slice(&self, offset: usize, length: usize) -> ArrayData

Creates a zero-copy slice of itself. This creates a new ArrayData with a different offset, len and a shifted null bitmap.

Panics

Panics if offset + length > self.len().

pub fn buffer<T: ArrowNativeType>(&self, buffer: usize) -> &[T]

Returns the buffer as a slice of type T starting at self.offset

Panics

This function panics if:

• the buffer is not byte-aligned with type T, or
• the datatype is Boolean (it corresponds to a bit-packed buffer where the offset is not applicable)

pub fn new_empty(data_type: &DataType) -> Self

Returns a new empty ArrayData valid for data_type.

pub fn validate(&self) -> Result<(), ArrowError>

“cheap” validation of an ArrayData. Ensures buffers are sufficiently sized to store len + offset total elements of data_type and performs other inexpensive consistency checks.

This check is “cheap” in the sense that it does not validate the contents of the buffers (e.g. that all offsets for UTF8 arrays are within the bounds of the values buffer).

See ArrayData::validate_full to validate fully the offset content and the validitiy of utf8 data

pub fn validate_full(&self) -> Result<(), ArrowError>

“expensive” validation that ensures:

1. Null count is correct
2. All offsets are valid
3. All String data is valid UTF-8
4. All dictionary offsets are valid

Does not (yet) check

1. Union type_ids are valid see #85 Note calls validate() internally

pub fn validate_values(&self) -> Result<(), ArrowError>

pub fn ptr_eq(&self, other: &Self) -> bool

Returns true if this ArrayData is equal to other, using pointer comparisons to determine buffer equality. This is cheaper than PartialEq::eq but may return false when the arrays are logically equal

pub fn into_builder(self) -> ArrayDataBuilder

Converts this ArrayData into an ArrayDataBuilder

Trait Implementations

impl Clone for ArrayData

fn clone(&self) -> ArrayData

Returns a copy of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ArrayData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<ArrayData> for ArrayDataBuilder

fn from(d: ArrayData) -> Self

Converts to this type from the input type.

impl PartialEq<ArrayData> for ArrayData

fn eq(&self, other: &Self) -> bool

This method tests for self and other values to be equal, and is used by ==.

const fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.