Skip to main content

Module detect

Module detect 

Source
Expand description

Codec auto-detection from column type and data distribution.

Analyzes up to the first 1024 values of a column to select the optimal codec chain. Called at flush time when ColumnCodec::Auto is configured.

Selection strategy:

  • Partitions ≥ 1024 values → cascading codecs (ALP, FastLanes, etc.)
  • Partitions < 1024 values → single-step codecs (Gorilla, Delta, etc.)
  • f64 with >95% ALP encodability → AlpFastLanesLz4
  • f64 with ≤95% ALP encodability → Gorilla (fallback)
  • i64 timestamps/counters → DeltaFastLanesLz4
  • Symbol columns → FastLanesLz4 (small integer IDs)

Functions§

detect_codec
Detect the optimal codec for a column based on its type and data.
detect_f64_codec
Detect the optimal codec for an f64 column by analyzing the data.
detect_i64_codec
Detect the optimal codec for an i64 column by analyzing the data.