Struct DeltaArray 

pub struct DeltaArray { /* private fields */ }

A FastLanes-style delta-encoded array of primitive values.

A DeltaArray comprises a sequence of chunks each representing 1,024 delta-encoded values, except the last chunk which may represent from one to 1,024 values.

Examples

use vortex_fastlanes::DeltaArray;
let array = DeltaArray::try_from_vec(vec![1_u32, 2, 3, 5, 10, 11]).unwrap();

Details

To facilitate slicing, this array accepts an offset and logical_len. The offset must be strictly less than 1,024 and the sum of offset and logical_len must not exceed the length of the deltas array. These values permit logical slicing without modifying any chunk containing a kept value. In particular, we may defer decompresison until the array is canonicalized or indexed. The offset is a physical offset into the first chunk, which necessarily contains 1,024 values. The logical_len is the number of logical values following the offset, which may be less than the number of physically stored values.

Each chunk is stored as a vector of bases and a vector of deltas. If the chunk physically contains 1,024 values, then there are as many bases as there are lanes of this type in a 1024-bit register. For example, for 64-bit values, there are 16 bases because there are 16 lanes. Each lane is a delta-encoding 1024 / bit_width long vector of values. The deltas are stored in the FastLanes order which splits the 1,024 values into one contiguous sub-sequence per-lane, thus permitting delta encoding.

If the chunk physically has fewer than 1,024 values, then it is stored as a traditional, non-SIMD-amenable, delta-encoded vector.

Note the validity is stored in the deltas array.

Implementations

impl DeltaArray

pub fn try_from_vec<T: NativePType>(vec: Vec<T>) -> VortexResult<Self>

pub fn try_from_primitive_array(array: &PrimitiveArray) -> VortexResult<Self>

pub fn try_from_delta_compress_parts( bases: ArrayRef, deltas: ArrayRef, ) -> VortexResult<Self>

Create a DeltaArray from the given bases and deltas arrays. Note the deltas might be nullable

pub fn try_new( bases: ArrayRef, deltas: ArrayRef, offset: usize, logical_len: usize, ) -> VortexResult<Self>

pub fn bases(&self) -> &ArrayRef

pub fn deltas(&self) -> &ArrayRef

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn dtype(&self) -> &DType

pub fn offset(&self) -> usize

The logical offset into the first chunk of Self::deltas.

Trait Implementations

impl AsRef<dyn Array> for DeltaArray

fn as_ref(&self) -> &dyn Array

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for DeltaArray

fn clone(&self) -> DeltaArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DeltaArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Deref for DeltaArray

type Target = dyn Array

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl From<DeltaArray> for ArrayRef

fn from(value: DeltaArray) -> ArrayRef

Converts to this type from the input type.

impl IntoArray for DeltaArray

fn into_array(self) -> ArrayRef

impl ValidityChildSliceHelper for DeltaArray

fn unsliced_child_and_slice(&self) -> (&ArrayRef, usize, usize)

fn sliced_child_array(&self) -> Arc<dyn Array>