Struct PrequentialEvaluator 

pub struct PrequentialEvaluator { /* private fields */ }

Prequential (test-then-train) evaluator for streaming regression models.

For each sample:

1. Predict – query the model before it sees this sample.
2. Record – update metrics with (target, prediction) if past warmup and on the evaluation step interval.
3. Train – feed the sample to the model.

This ensures the model is always evaluated on unseen data, which is the correct methodology for streaming ML (as opposed to train-then-test which leaks information).

Metric Backends

• Cumulative (RegressionMetrics) – always active, tracks MAE/MSE/RMSE/R-squared over all evaluated samples.
• Rolling (RollingRegressionMetrics) – optional, windowed metrics over the last N evaluated samples.
• EWMA (EwmaRegressionMetrics) – optional, exponentially weighted metrics with configurable span.

Example

use irithyll::evaluation::{PrequentialEvaluator, PrequentialConfig};
use irithyll::learner::{SGBTLearner, StreamingLearner};
use irithyll::SGBTConfig;

let config = SGBTConfig::builder()
    .n_steps(5)
    .learning_rate(0.1)
    .grace_period(10)
    .build()
    .unwrap();
let mut model = SGBTLearner::from_config(config);

let mut eval = PrequentialEvaluator::new()
    .with_rolling_window(50)
    .with_ewma(20);

// Simulate a data stream
for i in 0..200 {
    let x = [i as f64, (i as f64).sin()];
    let y = i as f64 * 0.5 + 1.0;
    eval.step(&mut model, &x, y);
}

assert!(eval.regression_metrics().n_samples() > 0);

Implementations

impl PrequentialEvaluator

pub fn new() -> Self

Create a new evaluator with default configuration (warmup=0, step_interval=1).

pub fn with_config(config: PrequentialConfig) -> Self

Create a new evaluator with the given configuration.

pub fn with_rolling_window(self, window_size: usize) -> Self

Enable rolling (windowed) regression metrics.

Metrics will be computed over the last window_size evaluated samples.

Panics

Panics if window_size is zero.

pub fn with_ewma(self, span: usize) -> Self

Enable EWMA (exponentially weighted) regression metrics.

The decay factor is alpha = 2 / (span + 1).

Panics

Panics if span is zero.

pub fn step( &mut self, model: &mut dyn StreamingLearner, features: &[f64], target: f64, ) -> f64

Process a single sample through the test-then-train protocol.

1. Predict using the current model state.
2. If past warmup and on the step interval, record metrics.
3. Train the model on this sample.
4. Return the prediction (made before training).

The model always trains on every sample, regardless of warmup or step interval. Only metric recording is gated.

pub fn evaluate<O: Observation>( &mut self, model: &mut dyn StreamingLearner, data: &[O], )

Run the full test-then-train protocol over a slice of observations.

Iterates over data, calling step for each observation.

pub fn regression_metrics(&self) -> &RegressionMetrics

Reference to the cumulative regression metrics.

pub fn rolling_metrics(&self) -> Option<&RollingRegressionMetrics>

Reference to the rolling regression metrics, if enabled.

pub fn ewma_metrics(&self) -> Option<&EwmaRegressionMetrics>

Reference to the EWMA regression metrics, if enabled.

pub fn samples_seen(&self) -> u64

Total number of samples processed (both warmup and evaluated).

pub fn samples_evaluated(&self) -> u64

Number of samples that were actually evaluated (past warmup, on step interval).

pub fn reset(&mut self)

Reset all metrics and counters to the initial state.

Configuration (warmup, step_interval, rolling window, EWMA span) is preserved.

Trait Implementations

impl Clone for PrequentialEvaluator

fn clone(&self) -> PrequentialEvaluator

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PrequentialEvaluator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for PrequentialEvaluator

fn default() -> Self

Returns the “default value” for a type.