Struct FSSTData 

pub struct FSSTData { /* private fields */ }

The inner data for an FSST-compressed array.

Holds the FSST symbol table (symbols + symbol_lengths) and the raw compressed codes bytes buffer. The codes offsets and validity live in the outer array’s slots (slots 1 and 2 respectively).

A full VarBinArray representing the codes can be reconstructed on demand via FSSTArrayExt::codes(), combining this buffer with the offsets/validity from slots.

Implementations

impl FSSTData

pub fn try_new( symbols: Buffer<Symbol>, symbol_lengths: Buffer<u8>, codes_bytes: BufferHandle, len: usize, ) -> VortexResult<Self>

Build FSST data from a set of symbols, symbol_lengths, and compressed codes bytes.

Symbols are 8-bytes and can represent short strings, each of which is assigned a code.

The codes_bytes buffer contains the concatenated compressed bytecodes for all elements. Each element’s compressed bytecodes are a sequence of 8-bit codes, where each code corresponds either to a symbol or to the “escape code” (which tells the decoder to emit the following byte without doing a table lookup).

The offsets and validity for the codes are stored in the array’s slots, not here. Use FSSTArrayExt::codes() to reconstruct a full VarBinArray.

pub fn validate( &self, dtype: &DType, len: usize, slots: &[Option<ArrayRef>], ) -> VortexResult<()>

pub fn len(&self) -> usize

Returns the number of elements in the array.

pub fn is_empty(&self) -> bool

Returns true if the array contains no elements.

pub fn symbols(&self) -> &Buffer<Symbol>

Access the symbol table array.

pub fn symbol_lengths(&self) -> &Buffer<u8>

Access the symbol lengths array.

pub fn codes_bytes_handle(&self) -> &BufferHandle

Access the compressed codes bytes buffer handle (may be on host or device).

pub fn codes_bytes(&self) -> &ByteBuffer

Access the compressed codes bytes on the host.

pub fn decompressor(&self) -> Decompressor<'_>

Build a Decompressor that can be used to decompress values from this array.

pub fn compressor(&self) -> &Compressor

Retrieves the FSST compressor.

Trait Implementations

impl ArrayEq for FSSTData

fn array_eq(&self, other: &Self, precision: Precision) -> bool

impl ArrayHash for FSSTData

fn array_hash<H: Hasher>(&self, state: &mut H, precision: Precision)

impl Clone for FSSTData

fn clone(&self) -> FSSTData

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FSSTData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for FSSTData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.