Struct FsstArray 

pub struct FsstArray { /* private fields */ }

In-memory FSST dictionary buffer that bundles compressed bytes, compact offsets, and the compressor needed to (de)compress values.

Implementations

impl FsstArray

pub fn train_compressor<'a>(input: impl Iterator<Item = &'a [u8]>) -> Compressor

Trains a compressor on a sequence of strings.

pub fn from_byte_array_with_compressor<T: ByteArrayType>( input: &GenericByteArray<T>, compressor: Arc<Compressor>, ) -> Self

Creates a new FSST buffer from a GenericByteArray and a compressor.

pub fn from_decimal128_array_with_compressor( array: &Decimal128Array, compressor: Arc<Compressor>, ) -> Self

Creates a new FSST buffer from a Decimal128Array and a compressor.

pub fn from_decimal256_array_with_compressor( array: &Decimal256Array, compressor: Arc<Compressor>, ) -> Self

Creates a new FSST buffer from a Decimal256Array and a compressor.

pub fn uncompressed_bytes(&self) -> usize

Returns the uncompressed byte size of this buffer.

pub fn get_array_memory_size(&self) -> usize

Returns the in-memory size of this buffer.

pub fn len(&self) -> usize

Returns the number of values in this buffer.

pub fn decompressor(&self) -> Decompressor<'_>

Returns a decompressor for this buffer.

pub fn compressor(&self) -> &Compressor

Returns a reference to the compressor.

pub fn compressor_arc(&self) -> Arc<Compressor>

Returns a clone of the shared compressor.

pub fn to_bytes(&self, out: &mut Vec<u8>)

Serializes this FSST buffer (raw bytes + compact offsets) to out.

pub fn from_bytes(bytes: Bytes, compressor: Arc<Compressor>) -> Self

Deserializes a FSST buffer from the to_bytes() format.

pub fn to_arrow_byte_array<T: ByteArrayType<Offset = i32>>( &self, ) -> GenericByteArray<T>

Decompress all values into an Arrow byte array.

pub fn to_decimal128_array( &self, data_type: &ArrowFixedLenByteArrayType, ) -> Decimal128Array

Converts this FSST buffer to a Decimal128Array.

pub fn to_decimal256_array( &self, data_type: &ArrowFixedLenByteArrayType, ) -> Decimal256Array

Converts this FSST buffer to a Decimal256Array.

impl FsstArray

pub fn to_uncompressed_selected( &self, selected: &[usize], ) -> (Buffer, OffsetBuffer<i32>)

Decompress the selected values into a buffer.

Trait Implementations

impl Clone for FsstArray

fn clone(&self) -> FsstArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FsstArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FsstBacking for FsstArray

fn uncompressed_bytes(&self) -> usize

Get the uncompressed bytes of the FSST buffer (used for sizing / squeeze bookkeeping).

fn get_array_memory_size(&self) -> usize

Get the in-memory size of the FSST backing (raw bytes + any in-memory indices).