Expand description
Bitpacking encodings
These encodings look for unused higher order bits and discard them. For example, if we have a u32 array and all values are between 0 and 5000 then we only need 12 bits to store each value. The encoding will discard the upper 20 bits and only store 12 bits.
This is a simple encoding that works well for data that has a small range.
In order to decode the values we need to know the bit width of the values. This can be stored inline with the data (miniblock) or in the encoding description, out of line (full zip).
The encoding is transparent because the output has a fixed width (just like the input) and we can easily jump to the correct value.
Structsยง
- Inline
Bitpacking - OutOf
Line Bitpacking - A transparent compressor that bit packs data