pub struct ArrayData { /* private fields */ }
Expand description
An generic representation of Arrow array data which encapsulates common attributes and
operations for Arrow array. Specific operations for different arrays types (e.g.,
primitive, list, struct) are implemented in Array
.
Implementations§
source§impl ArrayData
impl ArrayData
sourcepub unsafe fn new_unchecked(
data_type: DataType,
len: usize,
null_count: Option<usize>,
null_bit_buffer: Option<Buffer>,
offset: usize,
buffers: Vec<Buffer, Global>,
child_data: Vec<ArrayData, Global>
) -> ArrayData
pub unsafe fn new_unchecked(
data_type: DataType,
len: usize,
null_count: Option<usize>,
null_bit_buffer: Option<Buffer>,
offset: usize,
buffers: Vec<Buffer, Global>,
child_data: Vec<ArrayData, Global>
) -> ArrayData
Create a new ArrayData instance;
If null_count
is not specified, the number of nulls in
null_bit_buffer is calculated.
If the number of nulls is 0 then the null_bit_buffer
is set to None
.
Safety
The input values must form a valid Arrow array for
data_type
, or undefined behavior can results.
Note: This is a low level API and most users of the arrow
crate should create arrays using the methods in the array
module.
sourcepub fn try_new(
data_type: DataType,
len: usize,
null_bit_buffer: Option<Buffer>,
offset: usize,
buffers: Vec<Buffer, Global>,
child_data: Vec<ArrayData, Global>
) -> Result<ArrayData, ArrowError>
pub fn try_new(
data_type: DataType,
len: usize,
null_bit_buffer: Option<Buffer>,
offset: usize,
buffers: Vec<Buffer, Global>,
child_data: Vec<ArrayData, Global>
) -> Result<ArrayData, ArrowError>
Create a new ArrayData, validating that the provided buffers form a valid Arrow array of the specified data type.
If the number of nulls in null_bit_buffer
is 0 then the null_bit_buffer
is set to None
.
Note: This is a low level API and most users of the arrow
crate should create arrays using the methods in the array
module.
sourcepub const fn builder(data_type: DataType) -> ArrayDataBuilder
pub const fn builder(data_type: DataType) -> ArrayDataBuilder
Returns a builder to construct a ArrayData
instance.
sourcepub const fn data_type(&self) -> &DataType
pub const fn data_type(&self) -> &DataType
Returns a reference to the data type of this array data
sourcepub fn child_data(&self) -> &[ArrayData]
pub fn child_data(&self) -> &[ArrayData]
Returns a slice of children data arrays
sourcepub const fn null_bitmap(&self) -> Option<&Bitmap>
pub const fn null_bitmap(&self) -> Option<&Bitmap>
Returns a reference to the null bitmap of this array data
sourcepub fn null_buffer(&self) -> Option<&Buffer>
pub fn null_buffer(&self) -> Option<&Buffer>
Returns a reference to the null buffer of this array data.
pub const fn is_empty(&self) -> bool
sourcepub const fn null_count(&self) -> usize
pub const fn null_count(&self) -> usize
Returns the total number of nulls in this array
sourcepub fn get_buffer_memory_size(&self) -> usize
pub fn get_buffer_memory_size(&self) -> usize
Returns the total number of bytes of memory occupied by the buffers owned by this ArrayData.
sourcepub fn get_array_memory_size(&self) -> usize
pub fn get_array_memory_size(&self) -> usize
Returns the total number of bytes of memory occupied physically by this ArrayData.
sourcepub fn buffer<T>(&self, buffer: usize) -> &[T] ⓘwhere
T: ArrowNativeType,
pub fn buffer<T>(&self, buffer: usize) -> &[T] ⓘwhere
T: ArrowNativeType,
Returns the buffer
as a slice of type T
starting at self.offset
Panics
This function panics if:
- the buffer is not byte-aligned with type T, or
- the datatype is
Boolean
(it corresponds to a bit-packed buffer where the offset is not applicable)
sourcepub fn new_empty(data_type: &DataType) -> ArrayData
pub fn new_empty(data_type: &DataType) -> ArrayData
Returns a new empty ArrayData valid for data_type
.
sourcepub fn validate(&self) -> Result<(), ArrowError>
pub fn validate(&self) -> Result<(), ArrowError>
“cheap” validation of an ArrayData
. Ensures buffers are
sufficiently sized to store len
+ offset
total elements of
data_type
and performs other inexpensive consistency checks.
This check is “cheap” in the sense that it does not validate the contents of the buffers (e.g. that all offsets for UTF8 arrays are within the bounds of the values buffer).
See ArrayData::validate_full to validate fully the offset content and the validitiy of utf8 data
sourcepub fn validate_full(&self) -> Result<(), ArrowError>
pub fn validate_full(&self) -> Result<(), ArrowError>
“expensive” validation that ensures:
- Null count is correct
- All offsets are valid
- All String data is valid UTF-8
- All dictionary offsets are valid
Does not (yet) check
- Union type_ids are valid see #85
Note calls
validate()
internally
pub fn validate_values(&self) -> Result<(), ArrowError>
sourcepub fn ptr_eq(&self, other: &ArrayData) -> bool
pub fn ptr_eq(&self, other: &ArrayData) -> bool
Returns true if this ArrayData
is equal to other
, using pointer comparisons
to determine buffer equality. This is cheaper than PartialEq::eq
but may
return false when the arrays are logically equal
sourcepub fn into_builder(self) -> ArrayDataBuilder
pub fn into_builder(self) -> ArrayDataBuilder
Converts this ArrayData
into an ArrayDataBuilder
Trait Implementations§
source§impl From<ArrayData> for ArrayDataBuilder
impl From<ArrayData> for ArrayDataBuilder
source§fn from(d: ArrayData) -> ArrayDataBuilder
fn from(d: ArrayData) -> ArrayDataBuilder
source§impl From<ArrayData> for BooleanArray
impl From<ArrayData> for BooleanArray
source§fn from(data: ArrayData) -> BooleanArray
fn from(data: ArrayData) -> BooleanArray
source§impl<T> From<ArrayData> for DictionaryArray<T>where
T: ArrowPrimitiveType,
impl<T> From<ArrayData> for DictionaryArray<T>where
T: ArrowPrimitiveType,
Constructs a DictionaryArray
from an array data reference.
source§fn from(data: ArrayData) -> DictionaryArray<T>
fn from(data: ArrayData) -> DictionaryArray<T>
source§impl From<ArrayData> for FixedSizeBinaryArray
impl From<ArrayData> for FixedSizeBinaryArray
source§fn from(data: ArrayData) -> FixedSizeBinaryArray
fn from(data: ArrayData) -> FixedSizeBinaryArray
source§impl From<ArrayData> for FixedSizeListArray
impl From<ArrayData> for FixedSizeListArray
source§fn from(data: ArrayData) -> FixedSizeListArray
fn from(data: ArrayData) -> FixedSizeListArray
source§impl<T> From<ArrayData> for GenericByteArray<T>where
T: ByteArrayType,
impl<T> From<ArrayData> for GenericByteArray<T>where
T: ByteArrayType,
source§fn from(data: ArrayData) -> GenericByteArray<T>
fn from(data: ArrayData) -> GenericByteArray<T>
source§impl<OffsetSize> From<ArrayData> for GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
impl<OffsetSize> From<ArrayData> for GenericListArray<OffsetSize>where
OffsetSize: OffsetSizeTrait,
source§fn from(data: ArrayData) -> GenericListArray<OffsetSize>
fn from(data: ArrayData) -> GenericListArray<OffsetSize>
source§impl<T> From<ArrayData> for PrimitiveArray<T>where
T: ArrowPrimitiveType,
impl<T> From<ArrayData> for PrimitiveArray<T>where
T: ArrowPrimitiveType,
Constructs a PrimitiveArray
from an array data reference.