Struct Array 

pub struct Array<V: VTable> { /* private fields */ }

A typed owned handle to an array.

Array<V> holds an ArrayRef (shared, heap-allocated) and provides typed access to the encoding-specific data via Deref to V::TypedArrayData.

Buffers are intentionally not stored in the common ArrayInner or ArrayParts state. Encodings may expose buffers only when writing or serializing, and those buffers need not be the same representation they keep in memory. For example, an encoding may hold a deserialized in-memory data structure and synthesize serialized buffers at write time; hoisting buffers here would force it to retain both forms or make the serialized layout dictate the runtime layout.

This is the primary type for working with typed arrays. Convert to ArrayRef via into_array() or AsRef<ArrayRef>.

Implementations

impl<V: VTable> Array<V>

pub fn try_from_parts(new: ArrayParts<V>) -> VortexResult<Self>

Create a typed array from explicit construction parameters.

This is the safe construction path for encoding implementors. It calls VTable::validate before publishing the array as an ArrayRef.

pub fn try_from_array_ref(array: ArrayRef) -> Result<Self, ArrayRef>

Try to create a typed handle from an ArrayRef.

Returns the original ArrayRef in Err when the encoding id does not match V.

pub fn dtype(&self) -> &DType

Returns the logical dtype.

pub fn len(&self) -> usize

Returns the length.

pub fn is_empty(&self) -> bool

Returns whether this array is empty.

pub fn encoding_id(&self) -> ArrayId

Returns the encoding ID for V.

pub fn statistics(&self) -> StatsSetRef<'_>

Returns this array’s statistics set.

pub fn data(&self) -> &V::TypedArrayData

Returns a reference to the encoding-specific data.

pub fn data_mut(&mut self) -> Option<&mut V::TypedArrayData>

Try to fetch mutable access to the encoding-specific data.

Returns None when this handle is not the unique owner of the backing allocation.

pub fn try_into_parts(self) -> Result<ArrayParts<V>, Self>

Returns the full typed array construction parts if this handle owns the allocation.

pub fn with_stats_set(self, stats: StatsSet) -> Self

Replace the array’s statistics set and return the same typed handle.

pub fn into_data(self) -> V::TypedArrayData

Returns a clone of the inner encoding-specific data.

pub fn slots(&self) -> &[Option<ArrayRef>]

Returns the array slots.

pub fn as_array(&self) -> &ArrayRef

Returns the internal ArrayRef.

pub fn as_view(&self) -> ArrayView<'_, V>

Returns an ArrayView borrowing this array’s data.

impl<V: VTable> Array<V>

Public API methods that shadow DynArrayData / ArrayRef methods.

pub fn slice(&self, range: Range<usize>) -> VortexResult<ArrayRef>

Lazily or eagerly slice the array to range, depending on available kernels.

pub fn scalar_at(&self, index: usize) -> VortexResult<Scalar>

👎Deprecated:

Use execute_scalar instead, which allows passing an execution context for more efficient execution when fetching multiple scalars from the same array.

pub fn execute_scalar( &self, index: usize, ctx: &mut ExecutionCtx, ) -> VortexResult<Scalar>

Execute the array to extract a scalar at the given index.

pub fn filter(&self, mask: Mask) -> VortexResult<ArrayRef>

Filter the array with a selection mask.

pub fn take(&self, indices: ArrayRef) -> VortexResult<ArrayRef>

Gather rows from this array by index.

pub fn validity(&self) -> VortexResult<Validity>

Returns the array’s validity representation.

pub fn is_valid( &self, index: usize, ctx: &mut ExecutionCtx, ) -> VortexResult<bool>

Returns whether index is valid using the provided execution context.

pub fn is_invalid( &self, index: usize, ctx: &mut ExecutionCtx, ) -> VortexResult<bool>

Returns whether index is null using the provided execution context.

pub fn all_valid(&self, ctx: &mut ExecutionCtx) -> VortexResult<bool>

Returns whether every row is valid.

pub fn all_invalid(&self, ctx: &mut ExecutionCtx) -> VortexResult<bool>

Returns whether every row is null.

pub fn to_canonical(&self) -> VortexResult<Canonical>

👎Deprecated:

Use Array::<V>::execute::<Canonical>() instead

pub fn nbytes(&self) -> u64

Returns the estimated physical bytes owned or referenced by this array tree.

pub fn nbuffers(&self) -> usize

Returns the number of top-level buffers exposed by this encoding.

pub fn as_constant(&self) -> Option<Scalar>

Returns the scalar value when this array is known to be constant.

pub fn valid_count(&self, ctx: &mut ExecutionCtx) -> VortexResult<usize>

Counts valid rows, executing validity arrays when necessary.

pub fn invalid_count(&self, ctx: &mut ExecutionCtx) -> VortexResult<usize>

Counts null rows, executing validity arrays when necessary.

pub fn append_to_builder( &self, builder: &mut dyn ArrayBuilder, ctx: &mut ExecutionCtx, ) -> VortexResult<()>

Append this array’s logical values to a canonical builder.

impl Array<Bool>

Constructors and consuming methods for BoolArray.

pub fn new(bits: BitBuffer, validity: Validity) -> Self

Constructs a new BoolArray.

Panics

Panics if the validity length is not equal to the bit buffer length.

pub fn new_handle( handle: BufferHandle, offset: usize, len: usize, validity: Validity, ) -> Self

Constructs a new BoolArray from a BufferHandle.

Panics

Panics if the validity length is not equal to the bit buffer length.

pub fn try_new(bits: BitBuffer, validity: Validity) -> VortexResult<Self>

Constructs a new BoolArray.

Errors

Returns an error if the provided components do not satisfy the invariants.

pub fn try_new_from_handle( bits: BufferHandle, offset: usize, len: usize, validity: Validity, ) -> VortexResult<Self>

Build a new bool array from a BufferHandle, returning an error if the offset is too large or the buffer is not large enough to hold the values.

pub unsafe fn new_unchecked(bits: BitBuffer, validity: Validity) -> Self

Creates a new BoolArray without validation.

Safety

The caller must ensure that the validity length is equal to the bit buffer length.

pub fn validate(bits: &BitBuffer, validity: &Validity) -> VortexResult<()>

Validates the components that would be used to create a BoolArray.

pub fn from_indices<I: IntoIterator<Item = usize>>( length: usize, indices: I, validity: Validity, ) -> Self

Create a new BoolArray from a set of indices and a length.

All indices must be less than the length.

pub fn into_bit_buffer(self) -> BitBuffer

Returns the underlying BitBuffer of the array, consuming self.

impl Array<Bool>

pub fn patch( self, patches: &Patches, ctx: &mut ExecutionCtx, ) -> VortexResult<Self>

impl Array<Bool>

pub fn opt_bool_vec(&self, ctx: &mut ExecutionCtx) -> Vec<Option<bool>>

pub fn bool_vec(&self, ctx: &mut ExecutionCtx) -> Vec<bool>

impl Array<Chunked>

pub fn try_new(chunks: Vec<ArrayRef>, dtype: DType) -> VortexResult<Self>

Constructs a new ChunkedArray.

pub fn rechunk( &self, target_bytesize: u64, target_rowsize: usize, ) -> VortexResult<Self>

pub unsafe fn new_unchecked(chunks: Vec<ArrayRef>, dtype: DType) -> Self

Creates a new ChunkedArray without validation.

Safety

All chunks must have exactly the same DType as the provided dtype.

impl Array<Constant>

pub fn new<S>(scalar: S, len: usize) -> Self

where S: Into<Scalar>,

impl Array<Decimal>

pub fn into_data_parts(self) -> DecimalDataParts

impl Array<Decimal>

pub fn new<T: NativeDecimalType>( buffer: Buffer<T>, decimal_dtype: DecimalDType, validity: Validity, ) -> Self

Creates a new DecimalArray using a host-native buffer.

pub unsafe fn new_unchecked<T: NativeDecimalType>( buffer: Buffer<T>, decimal_dtype: DecimalDType, validity: Validity, ) -> Self

Creates a new DecimalArray without validation.

Safety

See DecimalData::new_unchecked.

pub fn try_new<T: NativeDecimalType>( buffer: Buffer<T>, decimal_dtype: DecimalDType, validity: Validity, ) -> VortexResult<Self>

Creates a new DecimalArray from a host-native buffer with validation.

pub fn from_iter<T: NativeDecimalType, I: IntoIterator<Item = T>>( iter: I, decimal_dtype: DecimalDType, ) -> Self

Creates a new DecimalArray from an iterator of values.

pub fn from_option_iter<T: NativeDecimalType, I: IntoIterator<Item = Option<T>>>( iter: I, decimal_dtype: DecimalDType, ) -> Self

Creates a new DecimalArray from an iterator of optional values.

pub fn new_handle( values: BufferHandle, values_type: DecimalType, decimal_dtype: DecimalDType, validity: Validity, ) -> Self

Creates a new DecimalArray from a BufferHandle.

pub fn try_new_handle( values: BufferHandle, values_type: DecimalType, decimal_dtype: DecimalDType, validity: Validity, ) -> VortexResult<Self>

Creates a new DecimalArray from a BufferHandle with validation.

pub unsafe fn new_unchecked_handle( values: BufferHandle, values_type: DecimalType, decimal_dtype: DecimalDType, validity: Validity, ) -> Self

Creates a new DecimalArray without validation from a BufferHandle.

Safety

See DecimalData::new_unchecked_handle.

pub fn patch( self, patches: &Patches, ctx: &mut ExecutionCtx, ) -> VortexResult<Self>

impl Array<Dict>

pub fn new(codes: ArrayRef, values: ArrayRef) -> Self

Build a new DictArray from its components, codes and values.

pub fn try_new(codes: ArrayRef, values: ArrayRef) -> VortexResult<Self>

Build a new DictArray from its components, codes and values.

pub unsafe fn new_unchecked(codes: ArrayRef, values: ArrayRef) -> Self

Build a new DictArray without validating the codes or values.

Safety

See DictData::new_unchecked.

pub unsafe fn set_all_values_referenced( self, all_values_referenced: bool, ) -> Self

Set whether all values in the dictionary are referenced by at least one code.

Safety

See DictData::set_all_values_referenced.

impl Array<Extension>

pub fn new(ext_dtype: ExtDTypeRef, storage_array: ArrayRef) -> Self

Constructs a new ExtensionArray.

Panics

Panics if the storage array is not compatible with the extension dtype.

pub fn try_new( ext_dtype: ExtDTypeRef, storage_array: ArrayRef, ) -> VortexResult<Self>

Tries to construct a new ExtensionArray.

pub fn try_new_from_vtable<V: ExtVTable>( vtable: V, metadata: V::Metadata, storage_array: ArrayRef, ) -> VortexResult<Self>

Creates a new ExtensionArray from a vtable, metadata, and a storage array.

impl Array<Filter>

pub fn new(array: ArrayRef, mask: Mask) -> Self

Creates a new FilterArray.

pub fn try_new(array: ArrayRef, mask: Mask) -> VortexResult<Self>

Constructs a new FilterArray.

impl Array<FixedSizeList>

pub fn new( elements: ArrayRef, list_size: u32, validity: Validity, len: usize, ) -> Self

Creates a new FixedSizeListArray.

pub fn try_new( elements: ArrayRef, list_size: u32, validity: Validity, len: usize, ) -> VortexResult<Self>

Constructs a new FixedSizeListArray.

pub unsafe fn new_unchecked( elements: ArrayRef, list_size: u32, validity: Validity, len: usize, ) -> Self

Creates a new FixedSizeListArray without validation.

Safety

See FixedSizeListData::new_unchecked.

pub fn into_data_parts(self) -> FixedSizeListDataParts

impl Array<Interleave>

pub fn try_new( values: Vec<ArrayRef>, array_indices: ArrayRef, row_indices: ArrayRef, ) -> VortexResult<Self>

Constructs a new InterleaveArray from values and the array_indices / row_indices selectors.

See the module docs for the full specification and invariants. The selectors must be non-nullable: they record a definite (array_index, row_index) per row, so null-predicate handling is the caller’s responsibility, resolved before the interleave is constructed. The per-row bounds on the selector values are a runtime precondition checked during execution.

pub unsafe fn new_unchecked( values: Vec<ArrayRef>, array_indices: ArrayRef, row_indices: ArrayRef, dtype: DType, ) -> Self

Constructs an InterleaveArray without re-validating the spec invariants.

This is the assembly half of try_new: it lays the selectors into slots[0]/slots[1] and the values into slots[2..] and stamps dtype, skipping the Interleave::check pass. It exists for hot internal paths — notably building the pushed-down validity interleave in ValidityVTable::validity — where the inputs are known to satisfy the invariants by construction and re-checking them is pure overhead.

Safety

The caller must uphold every module invariant: at least two values sharing a dtype up to nullability, both selectors non-nullable unsigned integers of equal length, and dtype equal to the value returned by Interleave::check for these arguments (the shared value type with the union of the values’ nullabilities).

pub unsafe fn new_unchecked_slots(slots: ArraySlots, dtype: DType) -> Self

Constructs an InterleaveArray from a pre-assembled slots buffer, skipping both the spec re-check and the slot copy that new_unchecked performs.

This is the lowest-level assembly path: it stamps dtype onto slots as-is. Callers that already own a correctly laid-out ArraySlots — for example reusing a validated parent’s selectors while swapping its values — avoid materializing an intermediate Vec<ArrayRef> and re-pushing it into a fresh ArraySlots.

Safety

The caller must uphold every module invariant and the slot layout: slots[0] is the array_indices selector, slots[1] is the row_indices selector, and slots[2..] are the value arrays. Every slot must be Some, there must be at least two values (slots.len() >= 4), and dtype must equal the value returned by Interleave::check for these arguments.

impl Array<List>

pub fn new(elements: ArrayRef, offsets: ArrayRef, validity: Validity) -> Self

Creates a new ListArray.

pub fn try_new( elements: ArrayRef, offsets: ArrayRef, validity: Validity, ) -> VortexResult<Self>

Constructs a new ListArray.

pub unsafe fn new_unchecked( elements: ArrayRef, offsets: ArrayRef, validity: Validity, ) -> Self

Creates a new ListArray without validation.

Safety

See ListData::new_unchecked.

pub fn into_data_parts(self) -> ListDataParts

impl Array<List>

pub fn from_iter_slow<O: IntegerPType, I: IntoIterator>( iter: I, dtype: Arc<DType>, ) -> VortexResult<ArrayRef>

where I::Item: IntoIterator, <I::Item as IntoIterator>::Item: Into<Scalar>,

This is a convenience method to create a list array from an iterator of iterators. This method is slow however since each element is first converted to a scalar and then appended to the array.

pub fn from_iter_opt_slow<O: IntegerPType, I: IntoIterator<Item = Option<T>>, T>( iter: I, dtype: Arc<DType>, ) -> VortexResult<ArrayRef>

where T: IntoIterator, T::Item: Into<Scalar>,

impl Array<ListView>

pub fn new( elements: ArrayRef, offsets: ArrayRef, sizes: ArrayRef, validity: Validity, ) -> Self

Creates a new ListViewArray.

pub fn try_new( elements: ArrayRef, offsets: ArrayRef, sizes: ArrayRef, validity: Validity, ) -> VortexResult<Self>

Constructs a new ListViewArray.

pub unsafe fn new_unchecked( elements: ArrayRef, offsets: ArrayRef, sizes: ArrayRef, validity: Validity, ) -> Self

Creates a new ListViewArray without validation.

Safety

See ListViewData::new_unchecked.

pub unsafe fn with_zero_copy_to_list(self, is_zctl: bool) -> Self

Mark whether this list view can be zero-copy converted to a list.

Safety

See ListViewData::with_zero_copy_to_list.

pub fn into_data_parts(self) -> ListViewDataParts

impl Array<ListView>

pub fn rebuild( &self, mode: ListViewRebuildMode, ctx: &mut ExecutionCtx, ) -> VortexResult<ListViewArray>

Rebuilds the ListViewArray according to the specified mode.

pub unsafe fn trim_elements( &self, start: usize, end: usize, ) -> VortexResult<ListViewArray>

Unsafely trims elements to the referenced half-open range [start, end), adjusting every offset down by start. The result preserves the original is_zero_copy_to_list flag.

SAFETY

start must be the minimum value of offsets, and end should be the maximum value of offsets[i] + size[i] over all indices i of offsets. Otherwise, offsets and sizes may hold references to elements that no longer exist and the array will be corrupted.

impl Array<Masked>

pub fn try_new(child: ArrayRef, validity: Validity) -> VortexResult<Self>

Constructs a new MaskedArray.

impl Array<Null>

pub fn new(len: usize) -> Self

impl Array<Primitive>

pub fn from_option_iter<T: NativePType, I: IntoIterator<Item = Option<T>>>( iter: I, ) -> Self

Create a PrimitiveArray from an iterator of T. NOTE: we cannot impl FromIterator trait since it conflicts with FromIterator<T>.

pub fn to_buffer<T: NativePType>(&self) -> Buffer<T>

Get a buffer in host memory holding all the values.

NOTE: some values may be nonsense if the validity buffer indicates that the value is null.

pub fn into_buffer<T: NativePType>(self) -> Buffer<T>

Consume the array and get a host Buffer containing the data values.

pub fn into_buffer_mut<T: NativePType>(self) -> BufferMut<T>

Extract a mutable buffer from the PrimitiveArray. Attempts to do this with zero-copy if the buffer is uniquely owned, otherwise will make a copy.

pub fn try_into_buffer_mut<T: NativePType>( self, ) -> Result<BufferMut<T>, Buffer<T>>

Try to extract a mutable buffer from the PrimitiveArray with zero copy.

impl Array<Primitive>

pub fn patch( self, patches: &Patches, ctx: &mut ExecutionCtx, ) -> VortexResult<Self>

impl Array<Primitive>

pub fn top_value(&self) -> VortexResult<Option<(PValue, usize)>>

Compute most common present value of this array

impl Array<Primitive>

pub fn empty<T: NativePType>(nullability: Nullability) -> Self

pub fn new<T: NativePType>( buffer: impl Into<Buffer<T>>, validity: Validity, ) -> Self

Creates a new PrimitiveArray.

Panics

Panics if the provided components do not satisfy the invariants.

pub fn try_new<T: NativePType>( buffer: Buffer<T>, validity: Validity, ) -> VortexResult<Self>

Constructs a new PrimitiveArray.

pub unsafe fn new_unchecked<T: NativePType>( buffer: Buffer<T>, validity: Validity, ) -> Self

Creates a new PrimitiveArray without validation.

Safety

See PrimitiveData::new_unchecked.

pub unsafe fn new_unchecked_from_handle( handle: BufferHandle, ptype: PType, validity: Validity, ) -> Self

Create a new array from a buffer handle.

Safety

See PrimitiveData::new_unchecked_from_handle.

pub fn from_buffer_handle( handle: BufferHandle, ptype: PType, validity: Validity, ) -> Self

Creates a new PrimitiveArray from a BufferHandle.

pub fn from_byte_buffer( buffer: ByteBuffer, ptype: PType, validity: Validity, ) -> Self

Creates a new PrimitiveArray from a ByteBuffer.

pub fn from_values_byte_buffer( valid_elems_buffer: ByteBuffer, ptype: PType, validity: Validity, n_rows: usize, ctx: &mut ExecutionCtx, ) -> Self

Create a PrimitiveArray from a byte buffer containing only the valid elements.

pub fn validate<T: NativePType>( buffer: &Buffer<T>, validity: &Validity, ) -> VortexResult<()>

Validates the components that would be used to create a PrimitiveArray.

pub fn into_data_parts(self) -> PrimitiveDataParts

pub fn map_each_with_validity<T, R, F>( self, ctx: &mut ExecutionCtx, f: F, ) -> VortexResult<Self>

where T: NativePType, R: NativePType, F: FnMut((T, bool)) -> R,

impl Array<ScalarFn>

pub fn try_new( scalar_fn: ScalarFnRef, children: Vec<ArrayRef>, ) -> VortexResult<Self>

Create a new ScalarFnArray from a scalar function and its children.

pub fn try_new_with_len( scalar_fn: ScalarFnRef, children: Vec<ArrayRef>, len: usize, ) -> VortexResult<Self>

Create a new ScalarFnArray from a scalar function, children, and an explicit length.

This is needed for zero-child scalar functions and deserialization paths where there is no child array to infer the length from.

impl Array<Shared>

pub fn new(source: ArrayRef) -> Self

Creates a new SharedArray.

impl Array<Slice>

pub fn try_new(child: ArrayRef, range: Range<usize>) -> VortexResult<Self>

Constructs a new SliceArray.

pub fn new(child: ArrayRef, range: Range<usize>) -> Self

Constructs a new SliceArray.

impl Array<Struct>

pub fn new( names: FieldNames, fields: impl Into<Arc<[ArrayRef]>>, length: usize, validity: Validity, ) -> Self

Creates a new StructArray.

pub fn try_new( names: FieldNames, fields: impl Into<Arc<[ArrayRef]>>, length: usize, validity: Validity, ) -> VortexResult<Self>

Constructs a new StructArray.

pub unsafe fn new_unchecked( fields: impl Into<Arc<[ArrayRef]>>, dtype: StructFields, length: usize, validity: Validity, ) -> Self

Creates a new StructArray without validation.

Safety

Caller must ensure the field arrays match the supplied dtype, length, and validity.

pub fn try_new_with_dtype( fields: impl Into<Arc<[ArrayRef]>>, dtype: StructFields, length: usize, validity: Validity, ) -> VortexResult<Self>

Constructs a new StructArray with an explicit dtype.

pub fn from_fields<N: AsRef<str>>(items: &[(N, ArrayRef)]) -> VortexResult<Self>

Construct a StructArray from named fields.

pub fn try_from_iter_with_validity<N: AsRef<str>, A: IntoArray, T: IntoIterator<Item = (N, A)>>( iter: T, validity: Validity, ) -> VortexResult<Self>

Create a StructArray from an iterator of (name, array) pairs with validity.

pub fn try_from_iter<N: AsRef<str>, A: IntoArray, T: IntoIterator<Item = (N, A)>>( iter: T, ) -> VortexResult<Self>

Create a StructArray from an iterator of (name, array) pairs.

pub fn project(&self, projection: &[FieldName]) -> VortexResult<Self>

Return a new StructArray with the given projection applied.

Projection does not copy data arrays. Projection is defined by an ordinal array slice which specifies the new ordering of columns in the struct. The projection can be used to perform column re-ordering, deletion, or duplication at a logical level, without any data copying.

pub fn new_fieldless_with_len(len: usize) -> Self

Create a fieldless StructArray with the given length.

pub fn into_data_parts(self) -> StructDataParts

pub fn remove_column( &self, name: impl Into<FieldName>, ) -> Option<(Self, ArrayRef)>

pub fn with_column( &self, name: impl Into<FieldName>, array: ArrayRef, ) -> VortexResult<Self>

pub fn remove_column_owned( &self, name: impl Into<FieldName>, ) -> Option<(Self, ArrayRef)>

pub fn try_concat<T>(chunks: impl IntoIterator<Item = T>) -> VortexResult<Self>

where T: Borrow<Array<Struct>>,

pub fn push_validity_into_children( &self, remove_struct_validity: bool, ) -> VortexResult<Self>

Push the struct’s top-level validity into each field, so a row null at the struct level becomes null in every field.

If remove_struct_validity is set the result is non-nullable; otherwise it keeps its top-level validity.

impl Array<VarBin>

Forwarding constructors for VarBinArray (= Array<VarBin>).

pub fn from_vec<T: AsRef<[u8]>>(vec: Vec<T>, dtype: DType) -> Self

pub fn from_iter<T: AsRef<[u8]>, I: IntoIterator<Item = Option<T>>>( iter: I, dtype: DType, ) -> Self

pub fn from_iter_nonnull<T: AsRef<[u8]>, I: IntoIterator<Item = T>>( iter: I, dtype: DType, ) -> Self

pub fn from_strs(value: Vec<&str>) -> Self

Create from a vector of string slices.

pub fn from_nullable_strs(value: Vec<Option<&str>>) -> Self

Create from a vector of optional string slices.

pub fn from_bytes(value: Vec<&[u8]>) -> Self

Create from a vector of byte slices.

pub fn from_nullable_bytes(value: Vec<Option<&[u8]>>) -> Self

Create from a vector of optional byte slices.

pub fn into_data_parts(self) -> VarBinDataParts

impl Array<VarBin>

pub fn new( offsets: ArrayRef, bytes: ByteBuffer, dtype: DType, validity: Validity, ) -> Self

Creates a new VarBinArray.

pub unsafe fn new_unchecked( offsets: ArrayRef, bytes: ByteBuffer, dtype: DType, validity: Validity, ) -> Self

Creates a new VarBinArray without validation.

Safety

See VarBinData::new_unchecked.

pub unsafe fn new_unchecked_from_handle( offsets: ArrayRef, bytes: BufferHandle, dtype: DType, validity: Validity, ) -> Self

Creates a new VarBinArray without validation from a BufferHandle.

Safety

See VarBinData::new_unchecked_from_handle.

pub fn try_new( offsets: ArrayRef, bytes: ByteBuffer, dtype: DType, validity: Validity, ) -> VortexResult<Self>

Constructs a new VarBinArray.

impl Array<VarBinView>

pub fn from_iter<T: AsRef<[u8]>, I: IntoIterator<Item = Option<T>>>( iter: I, dtype: DType, ) -> Self

Construct a VarBinViewArray from an iterator of optional byte slices.

pub fn from_iter_str<T: AsRef<str>, I: IntoIterator<Item = T>>(iter: I) -> Self

pub fn from_iter_nullable_str<T: AsRef<str>, I: IntoIterator<Item = Option<T>>>( iter: I, ) -> Self

pub fn from_iter_bin<T: AsRef<[u8]>, I: IntoIterator<Item = T>>(iter: I) -> Self

pub fn from_iter_nullable_bin<T: AsRef<[u8]>, I: IntoIterator<Item = Option<T>>>( iter: I, ) -> Self

pub fn try_new( views: Buffer<BinaryView>, buffers: Arc<[ByteBuffer]>, dtype: DType, validity: Validity, ) -> VortexResult<Self>

Creates a new VarBinViewArray.

pub unsafe fn new_unchecked( views: Buffer<BinaryView>, buffers: Arc<[ByteBuffer]>, dtype: DType, validity: Validity, ) -> Self

Creates a new VarBinViewArray without validation.

Safety

See VarBinViewData::new_unchecked.

pub fn new_handle( views: BufferHandle, buffers: Arc<[BufferHandle]>, dtype: DType, validity: Validity, ) -> Self

Creates a new VarBinViewArray with device or host memory.

pub unsafe fn new_handle_unchecked( views: BufferHandle, buffers: Arc<[BufferHandle]>, dtype: DType, validity: Validity, ) -> Self

Construct a new array from BufferHandles without validation.

Safety

See VarBinViewData::new_handle_unchecked.

pub fn into_data_parts(self) -> VarBinViewDataParts

impl Array<VarBinView>

pub fn compact_buffers(&self) -> VortexResult<VarBinViewArray>

Returns a compacted copy of the input array, where all wasted space has been cleaned up. This operation can be very expensive, in the worst case copying all existing string data into a new allocation.

After slicing/taking operations VarBinViewArrays can continue to hold references to buffers that are no longer visible. We detect when there is wasted space in any of the buffers, and if so, will aggressively compact all visible outlined string data into new buffers while keeping well-utilized buffers unchanged.

pub fn compact_with_threshold( &self, buffer_utilization_threshold: f64, ) -> VortexResult<VarBinViewArray>

Returns a compacted copy of the input array using selective buffer compaction.

This method analyzes each buffer’s utilization and applies one of three strategies:

• KeepFull (zero-copy): Well-utilized buffers are kept unchanged
• Slice (zero-copy): Buffers with contiguous ranges of used data are sliced to that range
• Rewrite: Poorly-utilized buffers have their data copied to new compact buffers

By preserving or slicing well-utilized buffers, compaction becomes zero-copy in many cases.

Arguments

• buffer_utilization_threshold - Threshold in range [0, 1]. Buffers with utilization below this value will be compacted. Use 0.0 for no compaction, 1.0 for aggressive compaction of any buffer with wasted space.

impl Array<Variant>

pub fn try_new( core_storage: ArrayRef, shredded: Option<ArrayRef>, ) -> VortexResult<Self>

Creates a new VariantArray with logical variant core storage and optional shredded storage.

core_storage must have DType::Variant, but it may use any Variant-typed physical encoding. See VariantArrayExt for the higher-level storage contract.

shredded, when present, must be row-aligned with core_storage and stores typed values for selected variant paths.

Trait Implementations

impl<V: VTable> AsRef<ArrayRef> for Array<V>

fn as_ref(&self) -> &ArrayRef

Converts this type into a shared reference of the (usually inferred) input type.

impl<V: VTable> Clone for Array<V>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<V: VTable> Debug for Array<V>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<V: VTable> Deref for Array<V>

type Target = <V as VTable>::TypedArrayData

The resulting type after dereferencing.

fn deref(&self) -> &V::TypedArrayData

Dereferences the value.

impl DictOwnedExt for Array<Dict>

fn into_parts(self) -> DictParts

impl From<Array<FixedSizeList>> for GroupedArray

fn from(groups: FixedSizeListArray) -> Self

Converts to this type from the input type.

impl From<Array<ListView>> for GroupedArray

fn from(groups: ListViewArray) -> Self

Converts to this type from the input type.

impl<V: VTable> From<Array<V>> for ArrayRef

fn from(value: Array<V>) -> ArrayRef

Converts to this type from the input type.

impl From<Vec<&[u8]>> for Array<VarBin>

fn from(value: Vec<&[u8]>) -> Self

Converts to this type from the input type.

impl From<Vec<&str>> for Array<VarBin>

fn from(value: Vec<&str>) -> Self

Converts to this type from the input type.

impl From<Vec<Option<&[u8]>>> for Array<VarBin>

fn from(value: Vec<Option<&[u8]>>) -> Self

Converts to this type from the input type.

impl From<Vec<Option<&str>>> for Array<VarBin>

fn from(value: Vec<Option<&str>>) -> Self

Converts to this type from the input type.

impl From<Vec<Option<String>>> for Array<VarBin>

fn from(value: Vec<Option<String>>) -> Self

Converts to this type from the input type.

impl From<Vec<Option<Vec<u8>>>> for Array<VarBin>

fn from(value: Vec<Option<Vec<u8>>>) -> Self

Converts to this type from the input type.

impl From<Vec<String>> for Array<VarBin>

fn from(value: Vec<String>) -> Self

Converts to this type from the input type.

impl From<Vec<Vec<u8>>> for Array<VarBin>

fn from(value: Vec<Vec<u8>>) -> Self

Converts to this type from the input type.

impl FromIterator<ArrayRef> for Array<Chunked>

fn from_iter<T: IntoIterator<Item = ArrayRef>>(iter: T) -> Self

Creates a value from an iterator.

impl<'a> FromIterator<Option<&'a [u8]>> for Array<VarBin>

fn from_iter<T: IntoIterator<Item = Option<&'a [u8]>>>(iter: T) -> Self

Creates a value from an iterator.

impl<'a> FromIterator<Option<&'a [u8]>> for Array<VarBinView>

fn from_iter<T: IntoIterator<Item = Option<&'a [u8]>>>(iter: T) -> Self

Creates a value from an iterator.

impl<'a> FromIterator<Option<&'a str>> for Array<VarBin>

fn from_iter<T: IntoIterator<Item = Option<&'a str>>>(iter: T) -> Self

Creates a value from an iterator.

impl<'a> FromIterator<Option<&'a str>> for Array<VarBinView>

fn from_iter<T: IntoIterator<Item = Option<&'a str>>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<Option<String>> for Array<VarBin>

fn from_iter<T: IntoIterator<Item = Option<String>>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<Option<String>> for Array<VarBinView>

fn from_iter<T: IntoIterator<Item = Option<String>>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<Option<Vec<u8>>> for Array<VarBin>

fn from_iter<T: IntoIterator<Item = Option<Vec<u8>>>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<Option<Vec<u8>>> for Array<VarBinView>

fn from_iter<T: IntoIterator<Item = Option<Vec<u8>>>>(iter: T) -> Self

Creates a value from an iterator.

impl<V: VTable> IntoArray for Array<V>

fn into_array(self) -> ArrayRef

Convert this value into the erased array handle used by generic APIs.

impl TryFrom<Array<Extension>> for TemporalData

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(ext: ExtensionArray) -> Result<Self, Self::Error>

Performs the conversion.

impl<V: VTable> TypedArrayRef<V> for Array<V>

fn to_owned(&self) -> Array<V>

Returns an owned Array<V> from the reference.