Struct VarBinArray 

pub struct VarBinArray { /* private fields */ }

Implementations

impl VarBinArray

pub fn new( offsets: ArrayRef, bytes: ByteBuffer, dtype: DType, validity: Validity, ) -> Self

Creates a new VarBinArray.

Panics

Panics if the provided components do not satisfy the invariants documented in VarBinArray::new_unchecked.

pub fn try_new( offsets: ArrayRef, bytes: ByteBuffer, dtype: DType, validity: Validity, ) -> VortexResult<Self>

Constructs a new VarBinArray.

See VarBinArray::new_unchecked for more information.

Errors

Returns an error if the provided components do not satisfy the invariants documented in VarBinArray::new_unchecked.

pub unsafe fn new_unchecked( offsets: ArrayRef, bytes: ByteBuffer, dtype: DType, validity: Validity, ) -> Self

Creates a new VarBinArray without validation from these components:

• offsets is an array of byte offsets into the bytes buffer.
• bytes is a buffer containing all the variable-length data concatenated.
• dtype specifies whether this contains UTF-8 strings or binary data.
• validity holds the null values.

Safety

The caller must ensure all of the following invariants are satisfied:

Offsets Requirements

• offsets must be a non-nullable integer array.
• offsets must contain at least 1 element (for empty array, it contains [0]).
• All values in offsets must be monotonically non-decreasing.
• The first value in offsets must be 0.
• No offset value may exceed bytes.len().

Type Requirements

• dtype must be exactly DType::Binary or DType::Utf8.
• If dtype is DType::Utf8, every byte slice bytes[offsets[i]..offsets[i+1]] must be valid UTF-8.
• dtype.is_nullable() must match the nullability of validity.

Validity Requirements

• If validity is Validity::Array, its length must exactly equal offsets.len() - 1.

pub fn offsets(&self) -> &ArrayRef

pub fn bytes(&self) -> &ByteBuffer

Access the value bytes child buffer

Note

Bytes child buffer is never sliced when the array is sliced so this can include values that are not logically present in the array. Users should prefer sliced_bytes unless they’re resolving values via the offset child array.

pub fn sliced_bytes(&self) -> ByteBuffer

Access value bytes child array limited to values that are logically present in the array unlike bytes.

pub fn from_vec<T: AsRef<[u8]>>(vec: Vec<T>, dtype: DType) -> Self

pub fn from_iter<T: AsRef<[u8]>, I: IntoIterator<Item = Option<T>>>( iter: I, dtype: DType, ) -> Self

pub fn from_iter_nonnull<T: AsRef<[u8]>, I: IntoIterator<Item = T>>( iter: I, dtype: DType, ) -> Self

pub fn offset_at(&self, index: usize) -> usize

Get value offset at a given index

Note: There’s 1 more offsets than the elements in the array, thus last offset is at array length index

Panics if index is out of bounds

pub fn bytes_at(&self, index: usize) -> ByteBuffer

Access value bytes at a given index

Will return buffer referencing underlying data without performing a copy

pub fn into_parts(self) -> (DType, ByteBuffer, ArrayRef, Validity)

Consumes self, returning a tuple containing the DType, the bytes array, the offsets array, and the validity.

Methods from Deref<Target = dyn Array>

pub fn display_values(&self) -> impl Display

Display logical values of the array

For example, an i16 typed array containing the first five non-negative integers is displayed as: [0i16, 1i16, 2i16, 3i16, 4i16].

Examples

let array = buffer![0_i16, 1, 2, 3, 4].into_array();
assert_eq!(
    format!("{}", array.display_values()),
    "[0i16, 1i16, 2i16, 3i16, 4i16]",
)

See also: Array::display_as, DisplayArrayAs, and DisplayOptions.

pub fn display_as(&self, options: DisplayOptions) -> impl Display

Display the array as specified by the options.

See DisplayOptions for examples.

pub fn display_tree(&self) -> impl Display

Display the tree of encodings of this array as an indented lists.

While some metadata (such as length, bytes and validity-rate) are included, the logical values of the array are not displayed. To view the logical values see Array::display_as and DisplayOptions.

Examples

let array = buffer![0_i16, 1, 2, 3, 4].into_array();
let expected = "root: vortex.primitive(i16, len=5) nbytes=10 B (100.00%)
  metadata: EmptyMetadata
  buffer (align=2): 10 B (100.00%)
";
assert_eq!(format!("{}", array.display_tree()), expected);

pub fn display_table(&self) -> impl Display

Display the array as a formatted table.

For struct arrays, displays a column for each field in the struct. For regular arrays, displays a single column with values.

Examples

let s = StructArray::from_fields(&[
    ("x", buffer![1, 2].into_array()),
    ("y", buffer![3, 4].into_array()),
]).unwrap().into_array();
let expected = "
┌──────┬──────┐
│  x   │  y   │
├──────┼──────┤
│ 1i32 │ 3i32 │
├──────┼──────┤
│ 2i32 │ 4i32 │
└──────┴──────┘".trim();
assert_eq!(format!("{}", s.display_table()), expected);

pub fn as_<V: VTable>(&self) -> &V::Array

Returns the array downcast to the given A.

pub fn as_opt<V: VTable>(&self) -> Option<&V::Array>

Returns the array downcast to the given A.

pub fn is<V: VTable>(&self) -> bool

Is self an array with encoding from vtable V.

pub fn is_constant(&self) -> bool

pub fn is_constant_opts(&self, cost: Cost) -> bool

pub fn as_constant(&self) -> Option<Scalar>

pub fn nbytes(&self) -> u64

Total size of the array in bytes, including all children and buffers.

pub fn try_to_mask_fill_null_false(&self) -> VortexResult<Mask>

Converts from a possible nullable boolean array. Null values are treated as false.

pub fn to_array_iterator(&self) -> impl ArrayIterator + 'static

Create an ArrayIterator over the array.

pub fn serialize( &self, ctx: &ArrayContext, options: &SerializeOptions, ) -> VortexResult<Vec<ByteBuffer>>

Serialize the array into a sequence of byte buffers that should be written contiguously. This function returns a vec to avoid copying data buffers.

Optionally, padding can be included to guarantee buffer alignment and ensure zero-copy reads within the context of an external file or stream. In this case, the alignment of the first byte buffer should be respected when writing the buffers to the stream or file.

The format of this blob is a sequence of data buffers, possible with prefixed padding, followed by a flatbuffer containing an fba::Array message, and ending with a little-endian u32 describing the length of the flatbuffer message.

pub fn to_array_stream(&self) -> impl ArrayStream + 'static

Create an ArrayStream over the array.

pub fn as_null_typed(&self) -> NullTyped<'_>

Downcasts the array for null-specific behavior.

pub fn as_bool_typed(&self) -> BoolTyped<'_>

Downcasts the array for bool-specific behavior.

pub fn as_primitive_typed(&self) -> PrimitiveTyped<'_>

Downcasts the array for primitive-specific behavior.

pub fn as_decimal_typed(&self) -> DecimalTyped<'_>

Downcasts the array for decimal-specific behavior.

pub fn as_utf8_typed(&self) -> Utf8Typed<'_>

Downcasts the array for utf8-specific behavior.

pub fn as_binary_typed(&self) -> BinaryTyped<'_>

Downcasts the array for binary-specific behavior.

pub fn as_struct_typed(&self) -> StructTyped<'_>

Downcasts the array for struct-specific behavior.

pub fn as_list_typed(&self) -> ListTyped<'_>

Downcasts the array for list-specific behavior.

pub fn as_extension_typed(&self) -> ExtensionTyped<'_>

Downcasts the array for extension-specific behavior.

Trait Implementations

impl ArrayAccessor<[u8]> for VarBinArray

fn with_iterator<F, R>(&self, f: F) -> VortexResult<R>

where F: for<'a> FnOnce(&mut dyn Iterator<Item = Option<&'a [u8]>>) -> R,

Iterate over each element of the array, in-order.

impl AsRef<dyn Array> for VarBinArray

fn as_ref(&self) -> &dyn Array

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for VarBinArray

fn clone(&self) -> VarBinArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VarBinArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Deref for VarBinArray

type Target = dyn Array

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl From<VarBinArray> for ArrayRef

fn from(value: VarBinArray) -> ArrayRef

Converts to this type from the input type.

impl From<Vec<&[u8]>> for VarBinArray

fn from(value: Vec<&[u8]>) -> Self

Converts to this type from the input type.

impl From<Vec<&str>> for VarBinArray

fn from(value: Vec<&str>) -> Self

Converts to this type from the input type.

impl From<Vec<String>> for VarBinArray

fn from(value: Vec<String>) -> Self

Converts to this type from the input type.

impl From<Vec<Vec<u8>>> for VarBinArray

fn from(value: Vec<Vec<u8>>) -> Self

Converts to this type from the input type.

impl<'a> FromIterator<Option<&'a [u8]>> for VarBinArray

fn from_iter<T: IntoIterator<Item = Option<&'a [u8]>>>(iter: T) -> Self

Creates a value from an iterator.

impl<'a> FromIterator<Option<&'a str>> for VarBinArray

fn from_iter<T: IntoIterator<Item = Option<&'a str>>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<Option<String>> for VarBinArray

fn from_iter<T: IntoIterator<Item = Option<String>>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<Option<Vec<u8>>> for VarBinArray

fn from_iter<T: IntoIterator<Item = Option<Vec<u8>>>>(iter: T) -> Self

Creates a value from an iterator.

impl IntoArray for VarBinArray

fn into_array(self) -> ArrayRef

impl OperatorEq for VarBinArray

fn operator_eq(&self, other: &Self) -> bool

impl OperatorHash for VarBinArray

fn operator_hash<H: Hasher>(&self, state: &mut H)

impl ValidityHelper for VarBinArray

fn validity(&self) -> &Validity