downsample 0.0.6

[![Crates.io](https://img.shields.io/crates/v/downsample.svg?label=downsample)](https://crates.io/crates/downsample)
[![docs.rs](https://docs.rs/downsample/badge.svg)](https://docs.rs/downsample)
[![pipeline status](https://gitlab.com/xMAC94x/downsample/badges/master/pipeline.svg)](https://gitlab.com/xMAC94x/downsample/commits/master)
[![coverage report](https://gitlab.com/xMAC94x/downsample/badges/master/coverage.svg?min_good=85&min_acceptable=70)](https://gitlab.com/xMAC94x/downsample/commits/master)
[![license](https://img.shields.io/crates/l/downsample)](https://gitlab.com/xMAC94x/downsample/blob/master/LICENSE-MIT)
[![dependencies](https://deps.rs/repo/gitlab/xMAC94x/downsample/status.svg)](https://deps.rs/repo/gitlab/xMAC94x/downsample)

# downsample

`no_std + alloc + no_deps` storage for keeping downsampled metric history over long periods of time.

Provides a method to track data points over long time, aggregating them eventually to keep storage constant.
Downsampling is configured with `Reducer<T>`, a plain function pointer that receives one aggregation window as up to two slices. 
The aggregation to the next level always happens when one full batch as configured is reached.

## Fixed-frequency usage

```rust
# #[cfg(feature = "reducers")]
# {
use downsample::{reducers, FixedFrequencyBuilder};
use core::num::NonZeroUsize;

let nz = |value| NonZeroUsize::new(value).unwrap();

// Store:
// - 100 raw samples
// - 59 samples aggregated from 100 raw samples each
// - 59 samples aggregated from 60 previous-level samples each
// - 23 samples aggregated from 60 previous-level samples each
let mut temperature = FixedFrequencyBuilder::new(nz(100))
    .level(nz(59), nz(100), reducers::average_f32())
    .level(nz(59), nz(60), reducers::median_f32())
    .level(nz(23), nz(60), reducers::average_f32())
    .build();

for value in 0..1000 {
    temperature.push(value as f32);
}

let hourly = temperature.storage().level(2).unwrap();
for value in hourly.iter_oldest_first() {
    let _ = value;
}

let storage = temperature.storage();
assert_eq!(storage.level_metadata(0).unwrap().storage_capacity(), 100 + 100);
assert_eq!(storage.level_metadata(1).unwrap().storage_capacity(), 59 + 60);
assert_eq!(storage.level_metadata(2).unwrap().storage_capacity(), 59 + 60);
assert_eq!(storage.level_metadata(3).unwrap().storage_capacity(), 23 + 0);
# }
```

## Custom reducers

Reducers are regular functions that can do any work to reduce a large set of data points into a single data point.

```rust
use downsample::{FixedFrequencyBuilder, Reducer};
use core::num::NonZeroUsize;

let nz = |value| NonZeroUsize::new(value).unwrap();

fn last(first: &[u32], second: &[u32]) -> u32 {
    second.last().or_else(|| first.last()).copied().unwrap_or_default()
}

let mut samples = FixedFrequencyBuilder::new(nz(16))
    .level(nz(8), nz(4), Reducer::new(last))
    .build();

samples.push(42);
```

Built-in reducers include:

- `reducers::average_f32()` and `reducers::average_f64()`
- integer averages for common integer widths
- `reducers::min::<T>()`
- `reducers::max::<T>()`
- `reducers::median::<T>()` for `Ord` values, returning the lower median for even windows
- `reducers::median_f32()` and `reducers::median_f64()`, ignoring `NaN` values unless all values are `NaN`
- `reducers::first::<T>()`
- `reducers::last::<T>()`

The built-in `reducers` module is enabled by the default `reducers` feature.
Disable default features if you only use custom `Reducer::new(...)` functions.

## Read/write split

`FixedWriter<T>` owns the writable storage and the reducer list. The read side is exposed as `FixedStorage<T>`.

```rust,ignore
// Continuing the fixed-frequency example above:
let storage = temperature.storage();
let metadata = storage.level_metadata(0).unwrap();
assert_eq!(metadata.capacity(), 100);
```

## Benchmarks

Representative local results from `cargo bench --bench mod -- --sample-size 10 --measurement-time 1 --warm-up-time 1`.
The time column uses Criterion's middle estimate.

| Benchmark | Work | Time | Throughput |
| --- | --- | ---: | ---: |
| `push/single_steady_state_average` | one fixed-frequency push | 4.23 ns | - |
| `push/batch/day_history_average/100000` | 100,000 pushes with day-history levels | 263.71 us | 379.20 Melem/s |
| `push/batch/dense_rollover_average/100000` | 100,000 pushes with frequent rollovers | 478.36 us | 209.05 Melem/s |
| `read/indexed_level_lookup` | 1,000,000 indexed level reads | 1.38 ms | 723.14 Melem/s |
| `reducers/median_f32/split_1024` | median over a wrapped 1,024-value window | 1.65 us | 620.27 Melem/s |

Recorded on an AMD Ryzen 9 5900X 12-Core Processor.

## Time-based direction

Time-series configuration uses caller-chosen integer ticks for time tracking. This is currently still "under construction".

```rust
# #[cfg(feature = "reducers")]
# {
use downsample::{reducers, TimeSeriesBuilder};
use core::num::NonZeroUsize;

let nz = |value| NonZeroUsize::new(value).unwrap();

// Here `u64` is nanoseconds, but it could also be milliseconds, hardware ticks,
// or any other integer unit chosen by the caller.
let writer = TimeSeriesBuilder::<u64, f32>::new(nz(128))
    .level(nz(60), 1_000_000_000, reducers::average_f32())
    .build();

assert_eq!(writer.storage().level_count(), 1);
# }
```

The time-series module currently contains the integer-tick storage and writer configuration. Bucket rollover and
aggregation policy still need to be completed.

## Limitations

- Fixed-frequency bucket sizes must be known when the object is built.
- Fixed-frequency downsample factors are whole numbers: e.g. 100Hz -> 25Hz -> 5Hz is possible with factor 4 and 5, but 100Hz -> 25Hz -> 10Hz isn't as factor 2.5 is no integer.
- This crate is still `0.0.x`; API changes are expected.