Skip to main content

Module downsample

Module downsample 

Source
Expand description

Time-bucket downsampling engine.

Groups data points into fixed-width time buckets and applies an aggregation function per bucket. This enables efficient data reduction for dashboards, alerting queries, and long-term storage.

§Two entry points

downsample takes a whole RecordBatch and returns every bucket in it. It is correct only when the batch contains all the rows for the buckets it covers.

StreamingDownsampler is the one to use over a chunked scan. Bucket boundaries do not line up with batch boundaries, so downsampling each batch independently emits the straddling interval twice — once per side — which a caller then sums, averages or plots as two points. The streaming form keeps one open bucket across batches and closes it only when a row from a later bucket arrives, so a bucket is emitted exactly once no matter how the input was chunked. Peak memory is one bucket’s accumulator, not one batch’s worth of rows.

Structs§

StreamingDownsampler
Boundary-safe downsampling over a sequence of time-ordered batches.

Functions§

downsample
Downsample a RecordBatch into fixed-width time buckets.