Struct arrow2::array::PrimitiveArray

source · [−]

pub struct PrimitiveArray<T: NativeType> { /* private fields */ }

Expand description

A PrimitiveArray is Arrow’s semantically equivalent of an immutable Vec<Option<T>> where T is NativeType (e.g. i32). It implements Array.

One way to think about a PrimitiveArray is (DataType, Arc<Vec<T>>, Option<Arc<Vec<u8>>>) where:

the first item is the array’s logical type
the second is the immutable values
the third is the immutable validity (whether a value is null or not as a bitmap).

The size of this struct is O(1), as all data is stored behind an std::sync::Arc.

Example

use arrow2::array::PrimitiveArray;
use arrow2::bitmap::Bitmap;
use arrow2::buffer::Buffer;

let array = PrimitiveArray::from([Some(1i32), None, Some(10)]);
assert_eq!(array.value(0), 1);
assert_eq!(array.iter().collect::<Vec<_>>(), vec![Some(&1i32), None, Some(&10)]);
assert_eq!(array.values_iter().copied().collect::<Vec<_>>(), vec![1, 0, 10]);
// the underlying representation
assert_eq!(array.values(), &Buffer::from(vec![1i32, 0, 10]));
assert_eq!(array.validity(), Some(&Bitmap::from([true, false, true])));

Implementations

source

impl<T: NativeType> PrimitiveArray<T>

source

pub fn try_new(
 data_type: DataType,
 values: Buffer<T>,
 validity: Option<Bitmap>
) -> Result<Self, Error>

The canonical method to create a PrimitiveArray out of its internal components.

Errors

This function errors iff:

The validity is not None and its length is different from values’s length
The data_type’s PhysicalType is not equal to [PhysicalType::Primitive(T::PRIMITIVE)]

source

pub fn to(self, data_type: DataType) -> Self

Returns a new PrimitiveArray with a different logical type.

This function is useful to assign a different DataType to the array. Used to change the arrays’ logical type (see example).

Example

use arrow2::array::Int32Array;
use arrow2::datatypes::DataType;

let array = Int32Array::from(&[Some(1), None, Some(2)]).to(DataType::Date32);
assert_eq!(
   format!("{:?}", array),
   "Date32[1970-01-02, None, 1970-01-03]"
);

Panics

Panics iff the data_type’s PhysicalType is not equal to [PhysicalType::Primitive(T::PRIMITIVE)]

source

pub fn from_vec(values: Vec<T>) -> Self

Creates a (non-null) PrimitiveArray from a vector of values. This function is O(1).

pub fn iter(&self) -> ZipValidity<'_, &T, Iter<'_, T>>ⓘNotable traits for ZipValidity<'a, T, I>`impl<'a, T, I: Iterator<Item = T>> Iterator for ZipValidity<'a, T, I> type Item = Option<T>;`

Returns an iterator over the values and validity, Option<&T>.

source

pub fn values_iter(&self) -> Iter<'_, T>

Returns an iterator of the values, &T, ignoring the arrays’ validity.

source

pub fn len(&self) -> usize

Returns the length of this array

source

pub fn values(&self) -> &Buffer<T>

The values Buffer. Values on null slots are undetermined (they can be anything).

source

pub fn validity(&self) -> Option<&Bitmap>

Returns the optional validity.

source

pub fn data_type(&self) -> &DataType

Returns the arrays’ DataType.

source

pub fn value(&self, i: usize) -> T

Returns the value at slot i.

Equivalent to self.values()[i]. The value of a null slot is undetermined (it can be anything).

Panic

This function panics iff i >= self.len.

source

pub unsafe fn value_unchecked(&self, i: usize) -> T

Returns the value at index i. The value on null slots is undetermined (it can be anything).

Safety

Caller must be sure that i < self.len()

source

pub fn slice(&self, offset: usize, length: usize) -> Self

Returns a clone of this PrimitiveArray sliced by an offset and length.

Implementation

This operation is O(1) as it amounts to increase two ref counts.

Examples

use arrow2::array::PrimitiveArray;

let array = PrimitiveArray::from_vec(vec![1, 2, 3]);
assert_eq!(format!("{:?}", array), "Int32[1, 2, 3]");
let sliced = array.slice(1, 1);
assert_eq!(format!("{:?}", sliced), "Int32[2]");
// note: `sliced` and `array` share the same memory region.

Panic

This function panics iff offset + length > self.len().

source

pub unsafe fn slice_unchecked(&self, offset: usize, length: usize) -> Self

Returns a clone of this PrimitiveArray sliced by an offset and length.

Implementation

This operation is O(1) as it amounts to increase two ref counts.

Safety

The caller must ensure that offset + length <= self.len().

source

pub fn with_validity(self, validity: Option<Bitmap>) -> Self

Returns this PrimitiveArray with a new validity.

Panics

This function panics iff validity.len() != self.len().

source

pub fn set_validity(&mut self, validity: Option<Bitmap>)

Sets the validity of this PrimitiveArray.

Panics

This function panics iff validity.len() != self.len().

source

pub fn with_values(self, values: Buffer<T>) -> Self

Returns this PrimitiveArray with new values.

Panics

This function panics iff values.len() != self.len().

source

pub fn set_values(&mut self, values: Buffer<T>)

Update the values of this PrimitiveArray.

Panics

This function panics iff values.len() != self.len().

source

pub fn apply_validity<F: FnOnce(Bitmap) -> Bitmap>(&mut self, f: F)

Applies a function f to the validity of this array.

This is an API to leverage clone-on-write

Panics

This function panics if the function f modifies the length of the Bitmap.

source

pub fn get_mut_values(&mut self) -> Option<&mut [T]>

Returns an option of a mutable reference to the values of this PrimitiveArray.

source

pub fn into_mut(self) -> Either<Self, MutablePrimitiveArray<T>>ⓘNotable traits for Either<L, R>`impl<L, R> Iterator for Either<L, R>where L: Iterator, R: Iterator<Item = <L as Iterator>::Item>, type Item = <L as Iterator>::Item;impl<L, R> Read for Either<L, R>where L: Read, R: Read,impl<L, R> Write for Either<L, R>where L: Write, R: Write,impl<L, R> Future for Either<L, R>where L: Future, R: Future<Output = <L as Future>::Output>, type Output = <L as Future>::Output;`

Try to convert this PrimitiveArray to a MutablePrimitiveArray via copy-on-write semantics.

A PrimitiveArray is backed by a Buffer and Bitmap which are essentially Arc<Vec<_>>. This function returns a MutablePrimitiveArray (via std::sync::Arc::get_mut) iff both values and validity have not been cloned / are unique references to their underlying vectors.

This function is primarily used to re-use memory regions.

source

pub fn new_empty(data_type: DataType) -> Self

Returns a new empty (zero-length) PrimitiveArray.

source

pub fn new_null(data_type: DataType, length: usize) -> Self

Returns a new PrimitiveArray where all slots are null / None.

source

pub fn from_values<I: IntoIterator<Item = T>>(iter: I) -> Self

Creates a (non-null) PrimitiveArray from an iterator of values.

Implementation

This does not assume that the iterator has a known length.

source

pub fn from_slice<P: AsRef<[T]>>(slice: P) -> Self

Creates a (non-null) PrimitiveArray from a slice of values.

Implementation

This is essentially a memcopy and is thus O(N)

source

pub fn from_trusted_len_values_iter<I: TrustedLen<Item = T>>(iter: I) -> Self

Creates a (non-null) PrimitiveArray from a TrustedLen of values.

Implementation

This does not assume that the iterator has a known length.

source

pub unsafe fn from_trusted_len_values_iter_unchecked<I: Iterator<Item = T>>(
iter: I
) -> Self

Creates a new PrimitiveArray from an iterator over values

Safety

The iterator must be TrustedLen. I.e. that size_hint().1 correctly reports its length.

source

pub fn from_trusted_len_iter<I: TrustedLen<Item = Option<T>>>(iter: I) -> Self

Creates a PrimitiveArray from a TrustedLen of optional values.

source

pub unsafe fn from_trusted_len_iter_unchecked<I: Iterator<Item = Option<T>>>(
iter: I
) -> Self

Creates a PrimitiveArray from an iterator of optional values.

Safety

The iterator must be TrustedLen. I.e. that size_hint().1 correctly reports its length.

source

pub fn boxed(self) -> Box<dyn Array>

Boxes self into a Box<dyn Array>.

source

pub fn arced(self) -> Arc<dyn Array>

Boxes self into a std::sync::Arc<dyn Array>.

source

pub fn new(
 data_type: DataType,
 values: Buffer<T>,
 validity: Option<Bitmap>
) -> Self

Alias for Self::try_new(..).unwrap().

Panics

This function errors iff:

The validity is not None and its length is different from values’s length
The data_type’s PhysicalType is not equal to PhysicalType::Primitive.

source

pub fn from_data(
 data_type: DataType,
 values: Buffer<T>,
 validity: Option<Bitmap>
) -> Self

Alias for Self::try_new(..).unwrap().

Trait Implementations

source

impl<T: NativeType> Array for PrimitiveArray<T>

source

fn as_any(&self) -> &dyn Any

Converts itself to a reference of Any, which enables downcasting to concrete types.

source

fn as_any_mut(&mut self) -> &mut dyn Any

Converts itself to a mutable reference of Any, which enables mutable downcasting to concrete types.

source

fn len(&self) -> usize

The length of the Array. Every array has a length corresponding to the number of elements (slots). Read more

source

fn data_type(&self) -> &DataType

The DataType of the Array. In combination with Array::as_any, this can be used to downcast trait objects (dyn Array) to concrete arrays. Read more

source

fn validity(&self) -> Option<&Bitmap>

The validity of the Array: every array has an optional Bitmap that, when available specifies whether the array slot is valid or not (null). When the validity is None, all slots are valid. Read more

source

fn slice(&self, offset: usize, length: usize) -> Box<dyn Array>

Slices the Array, returning a new Box<dyn Array>. Read more

source

unsafe fn slice_unchecked(&self, offset: usize, length: usize) -> Box<dyn Array>

Slices the Array, returning a new Box<dyn Array>. Read more

source

fn with_validity(&self, validity: Option<Bitmap>) -> Box<dyn Array>

Clones this Array with a new new assigned bitmap. Read more

source

fn to_boxed(&self) -> Box<dyn Array>

Clone a &dyn Array to an owned Box<dyn Array>.

source

fn is_empty(&self) -> bool

whether the array is empty

source

fn null_count(&self) -> usize

The number of null slots on this Array. Read more

source

fn is_null(&self, i: usize) -> bool

Returns whether slot i is null. Read more

source