Module resample 

Resampling: train/test splits, cross-validation, and grid and random search.

Splits are row indices, so they work with any Design. Random splits draw from StreamRng with a seed, so they replay exactly.

Structs

GridSearch

The result of grid_search: each candidate’s mean score, and the best (lowest) one.

Split

One train/test split: row indices into the data.

Functions

cross_validate

Fits model on each split’s training rows and scores its predictions on the test rows with score(y_test, predicted, test_design). Returns one score per split. Splits run in parallel (map_splits).

grid_search

Scores every candidate in candidates by cross_validate on the same splits, with make(candidate) building the model, and picks the lowest mean score (scores are losses: deviance, RMSE, CRPS).

group_k_fold

Grouped k-fold: every row of a group lands in the same fold, so a policy or an insured seen in training never appears in its test set. Groups are shuffled with seed and dealt into k folds.

k_fold

k-fold cross-validation: rows shuffled with seed, then cut into k folds of near-equal size; fold i is the test set of split i.

log_uniform

A draw log-uniform between low and high (both positive): uniform in orders of magnitude, for learning rates and penalties.

map_splits

Runs f on every split in parallel on the Rayon pool (the outer loop, as docs/architecture.md prescribes) and returns the results in split order: the first error in split order, if any fails. Each split’s work is independent and seeded on its own, so results do not depend on the number of threads.

random_search

Random search: draws n candidates with draw from stream 0 of seed, then scores them as grid_search does. With several hyperparameters, random candidates cover each one’s range far better than a grid of the same size (Bergstra and Bengio, 2012), which is what neural networks need. log_uniform draws scale parameters such as learning rates.

time_ordered

Time-ordered splits for forecasting: for each of the last n_test distinct periods t, train on rows with period before t and test on rows with period t. For a reserving triangle, the period is the calendar diagonal (origin + development), and this is the diagonal backtest of docs/design/models.md.

uniform_int

A draw uniform over low..=high, for layer widths and counts.