Module bitpacking 

Bitpacking encodings

These encodings look for unused higher order bits and discard them. For example, if we have a u32 array and all values are between 0 and 5000 then we only need 12 bits to store each value. The encoding will discard the upper 20 bits and only store 12 bits.

This is a simple encoding that works well for data that has a small range.

In order to decode the values we need to know the bit width of the values. This can be stored inline with the data (miniblock) or in the encoding description, out of line (full zip).

The encoding is transparent because the output has a fixed width (just like the input) and we can easily jump to the correct value.

Structs

InlineBitpacking

OutOfLineBitpacking

A transparent compressor that bit packs data

Constants

ELEMS_PER_CHUNK

Number of values encoded in each inline bitpacking chunk.