Struct RDEncoder 

pub struct RDEncoder { /* private fields */ }

Encoder for ALP-RD (“real doubles”) values.

The encoder calculates its parameters from a single sample of floating-point values, and then can be applied to many vectors.

ALP-RD uses the algorithm outlined in Section 3.4 of the paper. The crux of it is that the front (most significant) bits of many double vectors tend to be the same, i.e. most doubles in a vector often use the same exponent and front bits. Compression proceeds by finding the best prefix of up to 16 bits that can be collapsed into a dictionary of up to 8 elements. Each double can then be broken into the front/left L bits, which neatly bit-packs down to 1-3 bits per element (depending on the actual dictionary size). The remaining R bits naturally bit-pack.

In the ideal case, this scheme allows us to store a sequence of doubles in 49 bits-per-value.

Our implementation draws on the MIT-licensed C++ implementation provided by the original authors.

Implementations

impl RDEncoder

pub fn new<T>(sample: &[T]) -> RDEncoder

where T: ALPRDFloat,

Builds a new encoder from a sample of doubles.

Long samples are not read in full: past a few thousand values the search examines evenly spread contiguous runs instead, so this call costs about the same for a million values as for a few thousand. Values in the unexamined gaps still encode correctly — a left-part pattern the search never saw becomes an exception in Self::split.

Panics

Panics if sample is empty.

pub fn from_parts(right_bit_width: u8, codes: Vec<u16>) -> RDEncoder

Builds a new encoder from known parameters.

Panics

Panics if codes holds more than MAX_DICT_SIZE entries.

pub fn right_bit_width(&self) -> u8

Returns the bit-width of the right (least significant) part of each value.

pub fn left_bit_width(&self) -> u8

Returns the bit-width of the dictionary codes of the left (most significant) parts.

pub fn codes(&self) -> &[u16]

Returns the dictionary of left parts, indexed by code.

pub fn split<T>(&self, doubles: &[T]) -> Split<T, <T as ALPRDFloat>::UINT>

where T: ALPRDFloat,

Encodes the floating-point values into a Split.

Panics

Panics if the encoder holds no dictionary entries, which only an empty codes passed to Self::from_parts can produce.

pub fn split_parts<T>( &self, doubles: &[T], ) -> (Vec<u16>, Vec<<T as ALPRDFloat>::UINT>, Vec<u64>, Vec<u16>)

where T: ALPRDFloat,

Splits the floating-point values into their dictionary-encoded left parts, their right parts, and the positions and values of the left parts that are not in the dictionary.

The left parts are returned as dictionary codes, packable into Self::left_bit_width bits; the right parts are packable into Self::right_bit_width bits. Positions of exceptions hold a code of zero.

Panics

Panics if the encoder holds no dictionary entries, which only an empty codes passed to Self::from_parts can produce.

Trait Implementations

impl RDEncoderExt for RDEncoder

fn encode(&self, array: ArrayView<'_, Primitive>) -> ALPRDArray

Encode a set of floating point values with ALP-RD.