Struct Utf8Array 

pub struct Utf8Array<O>
where
    O: Offset,
{ /* private fields */ }

A Utf8Array is arrow’s semantic equivalent of an immutable Vec<Option<String>>. Cloning and slicing this struct is O(1).

Example

use arrow2::bitmap::Bitmap;
use arrow2::buffer::Buffer;
use arrow2::array::Utf8Array;
let array = Utf8Array::<i32>::from([Some("hi"), None, Some("there")]);
assert_eq!(array.value(0), "hi");
assert_eq!(array.iter().collect::<Vec<_>>(), vec![Some("hi"), None, Some("there")]);
assert_eq!(array.values_iter().collect::<Vec<_>>(), vec!["hi", "", "there"]);
// the underlying representation
assert_eq!(array.validity(), Some(&Bitmap::from([true, false, true])));
assert_eq!(array.values(), &Buffer::from(b"hithere".to_vec()));
assert_eq!(array.offsets(), &Buffer::from(vec![0, 2, 2, 2 + 5]));

Generic parameter

The generic parameter Offset can only be i32 or i64 and tradeoffs maximum array length with memory usage:

• the sum of lengths of all elements cannot exceed Offset::MAX
• the total size of the underlying data is array.len() * size_of::<Offset>() + sum of lengths of all elements

Safety

The following invariants hold:

• Two consecutives offsets casted (as) to usize are valid slices of values.
• A slice of values taken from two consecutives offsets is valid utf8.
• len is equal to validity.len(), when defined.

Implementations

impl<O> Utf8Array<O>

where O: Offset,

pub fn try_new( data_type: DataType, offsets: Buffer<O>, values: Buffer<u8>, validity: Option<Bitmap>, ) -> Result<Utf8Array<O>, Error>

Returns a Utf8Array created from its internal representation.

Errors

This function returns an error iff:

• the offsets are not monotonically increasing
• The last offset is not equal to the values’ length.
• the validity’s length is not equal to offsets.len() - 1.
• The data_type’s crate::datatypes::PhysicalType is not equal to either Utf8 or LargeUtf8.
• The values between two consecutive offsets are not valid utf8

Implementation

This function is O(N) - checking monotinicity and utf8 is O(N)

pub fn from_slice<T, P>(slice: P) -> Utf8Array<O>

where T: AsRef<str>, P: AsRef<[T]>,

Returns a Utf8Array from a slice of &str.

A convenience method that uses Self::from_trusted_len_values_iter.

pub fn from<T, P>(slice: P) -> Utf8Array<O>

where T: AsRef<str>, P: AsRef<[Option<T>]>,

Returns a new Utf8Array from a slice of &str.

A convenience method that uses Self::from_trusted_len_iter.

pub fn iter(&self) -> ZipValidity<'_, &str, ArrayValuesIter<'_, Utf8Array<O>>>

Returns an iterator of Option<&str>

pub fn values_iter(&self) -> ArrayValuesIter<'_, Utf8Array<O>>

Returns an iterator of &str

pub fn len(&self) -> usize

Returns the length of this array

pub fn value(&self, i: usize) -> &str

Returns the value of the element at index i, ignoring the array’s validity.

Panic

This function panics iff i >= self.len.

pub unsafe fn value_unchecked(&self, i: usize) -> &str

Returns the value of the element at index i, ignoring the array’s validity.

Safety

This function is safe iff i < self.len.

pub fn data_type(&self) -> &DataType

Returns the DataType of this array.

pub fn values(&self) -> &Buffer<u8>

Returns the values of this Utf8Array.

pub fn offsets(&self) -> &Buffer<O>

Returns the offsets of this Utf8Array.

pub fn validity(&self) -> Option<&Bitmap>

The optional validity.

pub fn slice(&self, offset: usize, length: usize) -> Utf8Array<O>

Returns a slice of this Utf8Array.

Implementation

This operation is O(1) as it amounts to essentially increase two ref counts.

Panic

This function panics iff offset + length >= self.len().

pub unsafe fn slice_unchecked( &self, offset: usize, length: usize, ) -> Utf8Array<O>

Returns a slice of this Utf8Array.

Implementation

This operation is O(1) as it amounts to essentially increase two ref counts.

Safety

The caller must ensure that offset + length <= self.len().

pub fn boxed(self) -> Box<dyn Array>

Boxes self into a Box<dyn Array>.

pub fn arced(self) -> Arc<dyn Array>

Boxes self into a std::sync::Arc<dyn Array>.

pub fn with_validity(self, validity: Option<Bitmap>) -> Utf8Array<O>

Returns this Utf8Array with a new validity.

Panics

This function panics iff validity.len() != self.len().

pub fn set_validity(&mut self, validity: Option<Bitmap>)

Sets the validity of this Utf8Array.

Panics

This function panics iff validity.len() != self.len().

pub fn into_mut(self) -> Either<Utf8Array<O>, MutableUtf8Array<O>>

Try to convert this Utf8Array to a MutableUtf8Array

pub fn new_empty(data_type: DataType) -> Utf8Array<O>

Returns a new empty Utf8Array.

The array is guaranteed to have no elements nor validity.

pub fn new_null(data_type: DataType, length: usize) -> Utf8Array<O>

Returns a new Utf8Array whose all slots are null / None.

pub fn default_data_type() -> DataType

Returns a default DataType of this array, which depends on the generic parameter O: DataType::Utf8 or DataType::LargeUtf8

pub unsafe fn try_new_unchecked( data_type: DataType, offsets: Buffer<O>, values: Buffer<u8>, validity: Option<Bitmap>, ) -> Result<Utf8Array<O>, Error>

Creates a new Utf8Array without checking for offsets monotinicity nor utf8-validity

Errors

This function returns an error iff:

• The last offset is not equal to the values’ length.
• the validity’s length is not equal to offsets.len() - 1.
• The data_type’s crate::datatypes::PhysicalType is not equal to either Utf8 or LargeUtf8.

Safety

This function is unsound iff:

• the offsets are not monotonically increasing
• The values between two consecutive offsets are not valid utf8

Implementation

This function is O(1)

pub fn new( data_type: DataType, offsets: Buffer<O>, values: Buffer<u8>, validity: Option<Bitmap>, ) -> Utf8Array<O>

Creates a new Utf8Array.

Panics

This function panics iff:

• the offsets are not monotonically increasing
• The last offset is not equal to the values’ length.
• the validity’s length is not equal to offsets.len() - 1.
• The data_type’s crate::datatypes::PhysicalType is not equal to either Utf8 or LargeUtf8.
• The values between two consecutive offsets are not valid utf8

Implementation

This function is O(N) - checking monotinicity and utf8 is O(N)

pub unsafe fn new_unchecked( data_type: DataType, offsets: Buffer<O>, values: Buffer<u8>, validity: Option<Bitmap>, ) -> Utf8Array<O>

Creates a new Utf8Array without checking for offsets monotinicity.

Errors

This function returns an error iff:

• The last offset is not equal to the values’ length.
• the validity’s length is not equal to offsets.len() - 1.
• The data_type’s crate::datatypes::PhysicalType is not equal to either Utf8 or LargeUtf8.

Safety

This function is unsound iff:

• the offsets are not monotonically increasing
• The values between two consecutive offsets are not valid utf8

Implementation

This function is O(1)

pub fn from_trusted_len_values_iter<T, I>(iterator: I) -> Utf8Array<O>

where T: AsRef<str>, I: TrustedLen<Item = T>,

Returns a (non-null) Utf8Array created from a TrustedLen of &str.

Implementation

This function is O(N)

pub fn from_iter_values<T, I>(iterator: I) -> Utf8Array<O>

where T: AsRef<str>, I: Iterator<Item = T>,

Creates a new Utf8Array from a Iterator of &str.

pub unsafe fn from_trusted_len_iter_unchecked<I, P>(iterator: I) -> Utf8Array<O>

where P: AsRef<str>, I: Iterator<Item = Option<P>>,

Creates a Utf8Array from an iterator of trusted length.

Safety

The iterator must be TrustedLen. I.e. that size_hint().1 correctly reports its length.

pub fn from_trusted_len_iter<I, P>(iterator: I) -> Utf8Array<O>

where P: AsRef<str>, I: TrustedLen<Item = Option<P>>,

Creates a Utf8Array from an iterator of trusted length.

pub unsafe fn try_from_trusted_len_iter_unchecked<E, I, P>( iterator: I, ) -> Result<Utf8Array<O>, E>

where P: AsRef<str>, I: IntoIterator<Item = Result<Option<P>, E>>,

Creates a Utf8Array from an falible iterator of trusted length.

Safety

The iterator must be TrustedLen. I.e. that size_hint().1 correctly reports its length.

pub fn try_from_trusted_len_iter<E, I, P>(iter: I) -> Result<Utf8Array<O>, E>

where P: AsRef<str>, I: TrustedLen<Item = Result<Option<P>, E>>,

Creates a Utf8Array from an fallible iterator of trusted length.

pub fn from_data( data_type: DataType, offsets: Buffer<O>, values: Buffer<u8>, validity: Option<Bitmap>, ) -> Utf8Array<O>

Alias for new

pub unsafe fn from_data_unchecked( data_type: DataType, offsets: Buffer<O>, values: Buffer<u8>, validity: Option<Bitmap>, ) -> Utf8Array<O>

Alias for Self::new_unchecked

Safety

This function is unsafe iff:

• the offsets are not monotonically increasing
• The values between two consecutive offsets are not valid utf8

Trait Implementations

impl<O> Array for Utf8Array<O>

where O: Offset,

fn as_any(&self) -> &(dyn Any + 'static)

Converts itself to a reference of Any, which enables downcasting to concrete types.

fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)

Converts itself to a mutable reference of Any, which enables mutable downcasting to concrete types.

fn len(&self) -> usize

The length of the Array. Every array has a length corresponding to the number of elements (slots).

fn data_type(&self) -> &DataType

The DataType of the Array. In combination with Array::as_any, this can be used to downcast trait objects (dyn Array) to concrete arrays.

fn validity(&self) -> Option<&Bitmap>

The validity of the Array: every array has an optional Bitmap that, when available specifies whether the array slot is valid or not (null). When the validity is None, all slots are valid.

fn slice(&self, offset: usize, length: usize) -> Box<dyn Array>

Slices the Array, returning a new Box<dyn Array>.

unsafe fn slice_unchecked(&self, offset: usize, length: usize) -> Box<dyn Array>

Slices the Array, returning a new Box<dyn Array>.

fn with_validity(&self, validity: Option<Bitmap>) -> Box<dyn Array>

Clones this Array with a new new assigned bitmap.

fn to_boxed(&self) -> Box<dyn Array>

Clone a &dyn Array to an owned Box<dyn Array>.

fn is_empty(&self) -> bool

whether the array is empty

fn null_count(&self) -> usize

The number of null slots on this Array.

fn is_null(&self, i: usize) -> bool

Returns whether slot i is null.

fn is_valid(&self, i: usize) -> bool

Returns whether slot i is valid.

impl ArrowArray for Utf8Array<i64>

impl<'a> ArrowGetItem for &'a Utf8Array<i64>

type Item = &'a str

fn get(&self, item: usize) -> Option<<&'a Utf8Array<i64> as ArrowGetItem>::Item>

unsafe fn get_unchecked( &self, item: usize, ) -> Option<<&'a Utf8Array<i64> as ArrowGetItem>::Item>

Safety

impl ChunkApplyKernel<Utf8Array<i64>> for ChunkedArray<Utf8Type>

fn apply_kernel( &self, f: &dyn Fn(&Utf8Array<i64>) -> Box<dyn Array>, ) -> ChunkedArray<Utf8Type>

Apply kernel and return result as a new ChunkedArray.

fn apply_kernel_cast<S>( &self, f: &dyn Fn(&Utf8Array<i64>) -> Box<dyn Array>, ) -> ChunkedArray<S>

where S: PolarsDataType,

Apply a kernel that outputs an array of different type.

impl<O> Clone for Utf8Array<O>

where O: Clone + Offset,

fn clone(&self) -> Utf8Array<O>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<O> Debug for Utf8Array<O>

where O: Offset,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<O> Default for Utf8Array<O>

where O: Offset,

fn default() -> Utf8Array<O>

Returns the “default value” for a type.

impl<'a, O> From<GrowableUtf8<'a, O>> for Utf8Array<O>

where O: Offset,

fn from(val: GrowableUtf8<'a, O>) -> Utf8Array<O>

Converts to this type from the input type.

impl<O> From<MutableUtf8Array<O>> for Utf8Array<O>

where O: Offset,

fn from(other: MutableUtf8Array<O>) -> Utf8Array<O>

Converts to this type from the input type.

impl<O> From<MutableUtf8ValuesArray<O>> for Utf8Array<O>

where O: Offset,

fn from(other: MutableUtf8ValuesArray<O>) -> Utf8Array<O>

Converts to this type from the input type.

impl FromDataUtf8 for Utf8Array<i64>

unsafe fn from_data_unchecked_default( offsets: Buffer<i64>, values: Buffer<u8>, validity: Option<Bitmap>, ) -> Utf8Array<i64>

Safety

impl<O, P> FromIterator<Option<P>> for Utf8Array<O>

where O: Offset, P: AsRef<str>,

fn from_iter<I>(iter: I) -> Utf8Array<O>

where I: IntoIterator<Item = Option<P>>,

Creates a value from an iterator.

impl<O> GenericBinaryArray<O> for Utf8Array<O>

where O: Offset,

fn values(&self) -> &[u8]

The values of the array

fn offsets(&self) -> &[O]

The offsets of the array

impl<'a, O> IntoIterator for &'a Utf8Array<O>

where O: Offset,

type Item = Option<&'a str>

The type of the elements being iterated over.

type IntoIter = ZipValidity<'a, &'a str, ArrayValuesIter<'a, Utf8Array<O>>>

Which kind of iterator are we turning this into?

fn into_iter(self) -> <&'a Utf8Array<O> as IntoIterator>::IntoIter

Creates an iterator from a value.

impl<O> PartialEq for Utf8Array<O>

where O: Offset,

fn eq(&self, other: &Utf8Array<O>) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl<O> PartialEq<&(dyn Array + 'static)> for Utf8Array<O>

where O: Offset,

fn eq(&self, other: &&(dyn Array + 'static)) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl<O> PartialEq<Utf8Array<O>> for &(dyn Array + 'static)

where O: Offset,

fn eq(&self, other: &Utf8Array<O>) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl ValueSize for Utf8Array<i64>

fn get_values_size(&self) -> usize

Useful for a Utf8 or a List to get underlying value size. During a rechunk this is handy