model-selection-rs 0.1.0

Cross-validation and model-selection utilities for Rust: stratified / group-aware / time-series splitting, nested CV, and learning & validation curves. Dependency-light, composes with any modeling crate.
Documentation

model-selection-rs

Cross-validation and model-selection utilities for Rust: stratified, group-aware, and time-aware splitting, nested cross-validation, and learning / validation curve utilities — a dependency-light crate that fills the specific gaps in smartcore::model_selection.

It composes with any modeling crate (smartcore, linfa, or hand-rolled models) through closures, and with any hyperparameter-search approach: this crate deliberately does not implement hyperparameter search itself.

[dependencies]
model-selection-rs = "0.1"

What it does

use ndarray::{Array1, Array2};
use model_selection_rs::evaluate::{cross_validate, BoxedScorer};
use model_selection_rs::scoring::{R2Score, MeanAbsoluteError};
use model_selection_rs::splitters::StratifiedKFold;

# fn demo(x: Array2<f64>, y: Array1<f64>, labels: Array1<i32>) {
// class-balanced 5-fold CV, two metrics in one pass
let skf = StratifiedKFold::new(5, &labels).unwrap();
let scorers: Vec<BoxedScorer> = vec![Box::new(R2Score), Box::new(MeanAbsoluteError)];
let res = cross_validate(&skf, &x, &y, |xt, yt| {
    // ... fit your model, return a closure that predicts ...
#     let m = yt.sum() / yt.len() as f64;
    move |xq: &Array2<f64>| Array1::from_elem(xq.nrows(), m)
}, &scorers, false).unwrap();

println!("R2  = {:.3} +/- {:.3}", res.mean_test_score(0), res.std_test_score(0));
println!("MAE = {:.3}", res.mean_test_score(1));
# }

See examples/ for one runnable program per feature: kfold_family, time_series_split, shuffle_split_family, cross_validate, nested_cv, learning_curve, validation_curve.

Comparison with sklearn.model_selection

Honest coverage map — what this crate provides, and what it deliberately leaves to other tools.

Splitters

scikit-learn this crate notes
KFold KFold optional seeded shuffle
StratifiedKFold StratifiedKFold generic label type; warns + adjusts on classes smaller than n_splits
GroupKFold GroupKFold greedy largest-group-first balancing; zero group leakage
StratifiedGroupKFold StratifiedGroupKFold greedy heuristic (documented approximation); exact group integrity, approximate class balance
TimeSeriesSplit TimeSeriesSplit expanding/fixed window, gap, test_size; positional order (no explicit timestamps)
ShuffleSplit ShuffleSplit count or fraction sizes
StratifiedShuffleSplit StratifiedShuffleSplit proportional per-class allocation
RepeatedKFold RepeatedKFold
RepeatedStratifiedKFold RepeatedStratifiedKFold
LeaveOneOut LeaveOneOut included for parity; documents its cost
LeavePOut, LeaveOneGroupOut, LeavePGroupsOut, PredefinedSplit — not implemented

Evaluation utilities

scikit-learn this crate notes
cross_validate / cross_val_score cross_validate multiple scorers in one pass; optional train scores & fit times; optional parallel feature
learning_curve learning_curve returns train and validation scores
validation_curve validation_curve score vs. one hyperparameter
nested CV (compose GridSearchCV in cross_val_score) nested_cross_validate tuning is a user closure — bring your own search
GridSearchCV, RandomizedSearchCV, HalvingGridSearchCV — out of scope — see below
cross_val_predict — not implemented

Scoring

scikit-learn this crate notes
accuracy_score Accuracy built-in
mean_absolute_error MeanAbsoluteError built-in
mean_squared_error MeanSquaredError built-in
RMSE RootMeanSquaredError built-in
r2_score R2Score built-in
f1_score smartcore_adapter::SmartcoreF1 behind smartcore-metrics feature
roc_auc_score smartcore_adapter::SmartcoreRocAuc behind smartcore-metrics feature
make_scorer make_scorer wrap any closure

What this crate deliberately doesn't do

Hyperparameter search. No grid search, random search, or Bayesian optimization lives here. That is the job of dedicated tools — manual grid/random loops, or a crate like tpe. NestedCV takes a tuning closure, so you plug in whichever search you like and this crate handles only the honest outer/inner evaluation structure around it. Keeping search out keeps the scope legible and avoids reinventing well-covered ground.

Feature flags

  • parallel — fan fold execution out over rayon in the evaluate utilities. Public signatures are unchanged; results are numerically identical to the serial path (parallelism only changes wall-clock time).
  • smartcore-metrics — enable the scoring::smartcore_adapter module wrapping smartcore::metrics (F1, ROC-AUC), for users who already depend on smartcore and want those without this crate reimplementing them.

Design notes

  • One trait for every splitter. All splitters implement a single CvSplitter trait returning (train, test) index pairs — never materialized data copies. Label- and group-aware splitters take their labels / groups at construction, which is what lets them satisfy the same trait as the unsupervised ones and be used uniformly by every evaluation utility.
  • KFold is reimplemented, not wrapped, so the crate has zero required dependency on smartcore. It is functionally equivalent to smartcore::model_selection::KFold.

MSRV & license

MSRV 1.74. Licensed under MIT.