Skip to main content

Module delta

Module delta 

Source
Expand description

Delta-of-delta timestamp encoding.

Exploits the regularity of time-series timestamps where successive deltas are often identical (constant sampling rate). The first value and first delta are stored raw; subsequent deltas-of-deltas are variable-length bit-packed.

§When to use delta-of-delta encoding

Ideal for: Monotonically increasing or nearly-monotonic sequences with a regular cadence, such as timestamps at a fixed sampling rate. In this case, the delta-of-delta is frequently zero, yielding compression ratios of 30× or higher (≈1 bit per value).

Acceptable for: Slowly changing integer sequences where consecutive deltas are small (e.g., incrementing counters). Compression will still be good as long as the delta-of-delta fits in the compact bit buckets.

Suboptimal for: Non-monotonic or highly variable data such as measured float values, gauge metrics, or random integers. Large swings produce large deltas-of-deltas that fall into the 68-bit fallback bucket, resulting in worse compression than raw storage (68 bits vs. 64 bits per value, plus header overhead).

Prefer instead:

  • Gorilla / Chimp XOR encoding for floating-point value columns — they exploit the bitwise similarity of consecutive IEEE 754 values.
  • RLE for columns with many consecutive repeated values (e.g., status codes, boolean flags).
  • IntegerEncoder (delta + ZigZag) for general integer columns that may not be monotonic.

The AdaptiveSelector in the unified encoder automatically samples data and picks the best encoding, so callers using ColumnEncoder do not need to make this choice manually.

§Encoding scheme

Delta-of-delta valuePrefix bitsValue bitsTotal bits
0001
−064..=631079
−256..=255110912
−2048..=204711101216
anything else11116468

Structs§

DeltaOfDeltaDecoder
Decodes delta-of-delta encoded timestamps.
DeltaOfDeltaEncoder
Encodes a sorted sequence of i64 timestamps using delta-of-delta encoding.