Module resample 

Resampling: train/test splits, cross-validation, and grid and random search.

Splits are row indices, so they work with any Design. Random splits draw from StreamRng with a seed, so they replay exactly.

Structs

GridSearch

The result of grid_search: each candidate’s mean score, and the best (lowest) one.

Split

One train/test split: row indices into the data.

Functions

cross_validate

Fits model on each split’s training rows and scores its predictions on the test rows with score(y_test, predicted, test_design). Returns one score per split. Splits run in parallel (map_splits).

grid_search

Scores every candidate in candidates by cross_validate on the same splits, with make(candidate) building the model, and picks the lowest mean score (scores are losses: deviance, RMSE, CRPS).

group_k_fold

Grouped k-fold: every row of a group lands in the same fold, so a policy or an insured seen in training never appears in its test set. Groups are shuffled with seed and dealt into k folds.

k_fold

k-fold cross-validation: rows shuffled with seed, then cut into k folds of near-equal size; fold i is the test set of split i.

log_uniform

A draw log-uniform between low and high (both positive): uniform in orders of magnitude, for learning rates and penalties.

map_splits

Runs f on every split in parallel on the Rayon pool (the outer loop, as docs/architecture.md prescribes) and returns the results in split order: the first error in split order, if any fails. Each split’s work is independent and seeded on its own, so results do not depend on the number of threads.

permutation

0..n shuffled by Fisher–Yates on stream stream of seed: the permutation that the model-agnostic explanations in the bindings apply to a column, so they shuffle alike in Python and R.

random_search

Random search: draws n candidates with draw from stream 0 of seed, then scores them as grid_search does. With several hyperparameters, random candidates cover each one’s range far better than a grid of the same size (Bergstra and Bengio, 2012), which is what neural networks need. log_uniform draws scale parameters such as learning rates.

time_ordered

Time-ordered splits for forecasting: for each of the last n_test distinct periods t, train on rows with period before t and test on rows with period t. For a reserving triangle, the period is the calendar diagonal (origin + development), and this is the diagonal backtest of docs/design/models.md.

uniform_int

A draw uniform over low..=high, for layer widths and counts.