Struct BitPackedData 

pub struct BitPackedData { /* private fields */ }

Implementations

impl BitPackedData

pub fn try_new( packed: BufferHandle, patches: Option<Patches>, bit_width: u8, offset: u16, ) -> VortexResult<Self>

Create a new bitpacked array using a buffer of packed data.

The packed data should be interpreted as a sequence of values with size bit_width.

Errors

This method returns errors if any of the metadata is inconsistent, for example the packed buffer provided does not have the right size according to the supplied length and target PType.

Safety

For signed arrays, it is the caller’s responsibility to ensure that there are no values that can be interpreted once unpacked to the provided PType.

This invariant is upheld by the compressor, but callers must ensure this if they wish to construct a new BitPackedArray from parts.

See also the encode method on this type for a safe path to create a new bit-packed array. A safe constructor for a BitPackedArray from its components:

• packed is ByteBuffer holding the compressed data that was packed with FastLanes bit-packing to a bit_width bits per value. length is the length of the original vector. Note that the packed is padded with zeros to the next multiple of 1024 elements if length is not divisible by 1024.
• ptype of the original data
• validity to track any nulls
• patches optionally provided for values that did not pack

Any failure in validation will result in an error.

Validation

• The ptype must be an integer
• validity must have length len
• Any patches must have any array_len equal to length
• The packed buffer must be exactly sized to hold length values of bit_width rounded up to the next multiple of 1024.

Any violation of these preconditions will result in an error.

pub fn ptype(&self, dtype: &DType) -> PType

pub fn packed(&self) -> &BufferHandle

Underlying bit packed values as byte array

pub fn packed_slice<T: NativePType + BitPacking>(&self) -> &[T]

Access the slice of packed values as an array of T

pub fn unpacked_chunks<T: BitPackedIter>( &self, dtype: &DType, len: usize, ) -> VortexResult<BitUnpackedChunks<T>>

Accessor for bit unpacked chunks

pub fn bit_width(&self) -> u8

Bit-width of the packed values

pub fn offset(&self) -> u16

pub fn encode( array: &ArrayRef, bit_width: u8, ctx: &mut ExecutionCtx, ) -> VortexResult<BitPackedArray>

Bit-pack an array of primitive integers down to the target bit-width using the FastLanes SIMD-accelerated packing kernels.

Errors

If the provided array is not an integer type, an error will be returned.

If the provided array contains negative values, an error will be returned.

If the requested bit-width for packing is larger than the array’s native width, an error will be returned.

pub fn max_packed_value(&self) -> usize

Calculate the maximum value that can be contained by this array, given its bit-width.

Note that this value need not actually be present in the array.

Trait Implementations

impl ArrayEq for BitPackedData

fn array_eq(&self, other: &Self, accuracy: EqMode) -> bool

impl ArrayHash for BitPackedData

fn array_hash<H: Hasher>(&self, state: &mut H, accuracy: EqMode)

impl Clone for BitPackedData

fn clone(&self) -> BitPackedData

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BitPackedData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for BitPackedData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.