Module alp 

ALP — Adaptive Lossless floating-Point compression (SIGMOD 2024).

Afroozeh, Kuffo & Boncz, “ALP: Adaptive Lossless floating-Point Compression”, Proc. ACM Manag. Data 2(1), SIGMOD 2024. The scheme DuckDB adopted, and the current state of the art for lossless f64 columns — better ratios than Chimp/Patas/Gorilla on real data and an order of magnitude faster to decode.

The idea

XOR-based codecs (Gorilla, Chimp, Patas) treat a double as an opaque bit pattern and hope consecutive values share a prefix. But most doubles in a time-series database never had 15 digits of precision to begin with: they are decimals that a sensor, meter or price feed produced with two or three fractional digits and that IEEE-754 then stored approximately. 231.45 is not a random 64-bit pattern — it is the integer 23145 with a known scale.

ALP recovers that integer. For a block it picks one exponent pair (e, f) and encodes each value as

i = round(v · 10^e · 10^-f)     decode: v = i · 10^f · 10^-e

The integers are then frame-of-reference coded and bit-packed, which is where the compression comes from: a power meter reporting 200–250 W to two decimals yields integers spanning 5000 values — 13 bits instead of 64.

Exactness

The transform is a guess, so every value is verified by decoding it again and comparing bit patterns. Values that do not round-trip — genuine high-precision doubles, NaN, ±inf, -0.0, anything outside i64 — are stored verbatim as exceptions alongside their positions. Both sides call the same decode_value, so encoder and decoder cannot disagree about what a given integer means, and the codec is bitwise lossless like every other encoder in this crate.

Scope

Only the decimal scheme is implemented. The paper’s second scheme, ALP_RD, targets columns of genuinely non-decimal doubles (scientific measurements, hashes) by splitting the bit pattern into a dictionary-coded left part and a packed right part. Chronix already has three XOR codecs that cover exactly that shape, and AdaptiveSelector picks between them by trial encoding — so ALP joins the competition and loses gracefully on data it is not for, rather than duplicating a fallback that already exists.

Wire format

[count: u32 LE][e: u8][f: u8][bit_width: u8][reference: i64 LE]
[exception_count: u32 LE]
[packed offsets: count × bit_width bits, MSB-first]
[exceptions: exception_count × ([position: u32 LE][value: 8 bytes LE])]

Structs

AlpDecoder

ALP float decoder.

AlpEncoder

ALP float encoder (SIGMOD 2024).

Functions

decode_value

Reconstruct the original value from its integer encoding.