Skip to main content

nested_cross_validate

Function nested_cross_validate 

Source
pub fn nested_cross_validate<OS, IS, P, Tune, Fit, M>(
    outer: &OS,
    inner: &IS,
    x: &Array2<f64>,
    y: &Array1<f64>,
    tune_fn: Tune,
    fit_fn: Fit,
    scorer: &(dyn Scorer + Sync),
) -> Result<NestedCvResults<P>>
where OS: CvSplitter, IS: CvSplitter + Sync, Tune: Fn(&Array2<f64>, &Array1<f64>, &IS) -> P + Sync, Fit: Fn(&P, &Array2<f64>, &Array1<f64>) -> M + Sync, M: Fn(&Array2<f64>) -> Array1<f64>, P: Send,
Expand description

Run nested cross-validation.

For each outer fold:

  1. tune_fn receives the outer-training data and the inner splitter, runs whatever hyperparameter search it likes (grid, tpe, …) using its own inner-CV loop, and returns the best hyperparameters P.
  2. fit_fn refits a final model with those hyperparameters on the full outer-training portion.
  3. that model is scored once on the untouched outer-test portion.

The inner loop never sees the outer-test data, so the outer scores carry none of the optimistic bias that tuning-and-evaluating on the same data produces. tune_fn is kept agnostic to how tuning happens, so this composes with any search approach rather than reimplementing one.

With the parallel feature the outer folds run concurrently over rayon.

§Errors

Propagates any error from outer.split(x.nrows()).