Expand description
Chimp float encoding (VLDB 2022).
Chimp improves upon Gorilla by exploiting the observation that XOR values of successive floating-point values cluster their leading zeros. It uses a leading-zero bucket scheme to reduce the overhead of encoding the number of leading zeros.
§Leading-zero buckets
| Bucket | Range (leading zeros) | Stored as |
|---|---|---|
| 0 | 0–7 | 3 bits |
| 1 | 8–15 | 3 bits |
| 2 | 16–23 | 3 bits |
| 3 | 24–31 | 3 bits |
| 4 | 32–39 | 3 bits |
| 5 | 40–47 | 3 bits |
| 6 | 48–55 | 3 bits |
| 7 | 56–63 | 3 bits |
§Chimp128 ring-buffer optimisation
Chimp128Encoder / Chimp128Decoder implement the Chimp128 variant
from the same VLDB 2022 paper. A ring buffer of the last 128 raw f64
bit-patterns is maintained. For each new value the encoder computes the
XOR against the previous value and against every ring-buffer entry,
picking whichever reference yields the most leading zeros. When a
ring-buffer reference wins, a 7-bit index is stored so the decoder can
reconstruct the same XOR base.
This improves compression by 5–15 % on highly periodic workloads (e.g. fixed-interval sensor data that cycles through a limited set of levels) at the cost of ~1 KiB of encoder/decoder state.
Structs§
- Chimp128
Decoder - Chimp128 float decoder.
- Chimp128
Encoder - Chimp128 float encoder — ring-buffer variant (VLDB 2022, §4.2).
- Chimp
Decoder - Chimp float decoder.
- Chimp
Encoder - Chimp float encoder (VLDB 2022).