lance_encoding::encoder

Trait FixedPerValueCompressor

source 
pub trait FixedPerValueCompressor:
    Debug
    + Send
    + Sync {
    // Required method
    fn compress(
        &self,
        data: DataBlock,
    ) -> Result<(FixedWidthDataBlock, ArrayEncoding)>;
}
Expand description

Trait for compression algorithms that are suitable for use in the zipped structural encoding

Compared to VariablePerValueCompressor, these compressors are capable of compressing the data so that every value has the exact same number of bits per value. For example, this is useful for encoding vector embeddings where every value has a fixed size but the values themselves are too large to use mini-block.

The advantage of a fixed-bytes-per-value is that we can do random access in 1 IOP instead of 2 and do not need a repetition index.

Required Methods§

source

fn compress( &self, data: DataBlock, ) -> Result<(FixedWidthDataBlock, ArrayEncoding)>

Compress the data into a single buffer where each value is encoded with the same number of bits

Also returns a description of the compression that can be used to decompress when reading the data back

Implementors§