model-selection-rs 0.1.0

# model-selection-rs

Cross-validation and model-selection utilities for Rust: **stratified**,
**group-aware**, and **time-aware** splitting, **nested** cross-validation, and
**learning / validation curve** utilities — a dependency-light crate that fills
the specific gaps in [`smartcore::model_selection`].

It composes with any modeling crate (`smartcore`, `linfa`, or hand-rolled
models) through closures, and with any hyperparameter-search approach: **this
crate deliberately does not implement hyperparameter search itself.**

```toml
[dependencies]
model-selection-rs = "0.1"
```

## What it does

```rust
use ndarray::{Array1, Array2};
use model_selection_rs::evaluate::{cross_validate, BoxedScorer};
use model_selection_rs::scoring::{R2Score, MeanAbsoluteError};
use model_selection_rs::splitters::StratifiedKFold;

# fn demo(x: Array2<f64>, y: Array1<f64>, labels: Array1<i32>) {
// class-balanced 5-fold CV, two metrics in one pass
let skf = StratifiedKFold::new(5, &labels).unwrap();
let scorers: Vec<BoxedScorer> = vec![Box::new(R2Score), Box::new(MeanAbsoluteError)];
let res = cross_validate(&skf, &x, &y, |xt, yt| {
    // ... fit your model, return a closure that predicts ...
#     let m = yt.sum() / yt.len() as f64;
    move |xq: &Array2<f64>| Array1::from_elem(xq.nrows(), m)
}, &scorers, false).unwrap();

println!("R2  = {:.3} +/- {:.3}", res.mean_test_score(0), res.std_test_score(0));
println!("MAE = {:.3}", res.mean_test_score(1));
# }
```

See [`examples/`](examples) for one runnable program per feature:
`kfold_family`, `time_series_split`, `shuffle_split_family`, `cross_validate`,
`nested_cv`, `learning_curve`, `validation_curve`.

## Comparison with `sklearn.model_selection`

Honest coverage map — what this crate provides, and what it deliberately leaves
to other tools.

### Splitters

| scikit-learn | this crate | notes |
|---|---|---|
| `KFold` | `KFold` | optional seeded shuffle |
| `StratifiedKFold` | `StratifiedKFold` | generic label type; warns + adjusts on classes smaller than `n_splits` |
| `GroupKFold` | `GroupKFold` | greedy largest-group-first balancing; zero group leakage |
| `StratifiedGroupKFold` | `StratifiedGroupKFold` | greedy heuristic (documented approximation); exact group integrity, approximate class balance |
| `TimeSeriesSplit` | `TimeSeriesSplit` | expanding/fixed window, `gap`, `test_size`; positional order (no explicit timestamps) |
| `ShuffleSplit` | `ShuffleSplit` | count or fraction sizes |
| `StratifiedShuffleSplit` | `StratifiedShuffleSplit` | proportional per-class allocation |
| `RepeatedKFold` | `RepeatedKFold` | |
| `RepeatedStratifiedKFold` | `RepeatedStratifiedKFold` | |
| `LeaveOneOut` | `LeaveOneOut` | included for parity; documents its cost |
| `LeavePOut`, `LeaveOneGroupOut`, `LeavePGroupsOut`, `PredefinedSplit` | — | not implemented |

### Evaluation utilities

| scikit-learn | this crate | notes |
|---|---|---|
| `cross_validate` / `cross_val_score` | `cross_validate` | multiple scorers in one pass; optional train scores & fit times; optional `parallel` feature |
| `learning_curve` | `learning_curve` | returns train **and** validation scores |
| `validation_curve` | `validation_curve` | score vs. one hyperparameter |
| nested CV (compose `GridSearchCV` in `cross_val_score`) | `nested_cross_validate` | tuning is a user closure — bring your own search |
| `GridSearchCV`, `RandomizedSearchCV`, `HalvingGridSearchCV` | — | **out of scope** — see below |
| `cross_val_predict` | — | not implemented |

### Scoring

| scikit-learn | this crate | notes |
|---|---|---|
| `accuracy_score` | `Accuracy` | built-in |
| `mean_absolute_error` | `MeanAbsoluteError` | built-in |
| `mean_squared_error` | `MeanSquaredError` | built-in |
| RMSE | `RootMeanSquaredError` | built-in |
| `r2_score` | `R2Score` | built-in |
| `f1_score` | `smartcore_adapter::SmartcoreF1` | behind `smartcore-metrics` feature |
| `roc_auc_score` | `smartcore_adapter::SmartcoreRocAuc` | behind `smartcore-metrics` feature |
| `make_scorer` | `make_scorer` | wrap any closure |

## What this crate deliberately doesn't do

**Hyperparameter search.** No grid search, random search, or Bayesian
optimization lives here. That is the job of dedicated tools — manual grid/random
loops, or a crate like [`tpe`](https://crates.io/crates/tpe). `NestedCV` takes a
*tuning closure*, so you plug in whichever search you like and this crate handles
only the honest outer/inner evaluation structure around it. Keeping search out
keeps the scope legible and avoids reinventing well-covered ground.

## Feature flags

- `parallel` — fan fold execution out over [`rayon`] in the `evaluate`
  utilities. Public signatures are unchanged; results are numerically identical
  to the serial path (parallelism only changes wall-clock time).
- `smartcore-metrics` — enable the `scoring::smartcore_adapter` module wrapping
  `smartcore::metrics` (F1, ROC-AUC), for users who already depend on
  `smartcore` and want those without this crate reimplementing them.

## Design notes

- **One trait for every splitter.** All splitters implement a single
  `CvSplitter` trait returning `(train, test)` **index** pairs — never
  materialized data copies. Label- and group-aware splitters take their labels /
  groups at construction, which is what lets them satisfy the same trait as the
  unsupervised ones and be used uniformly by every evaluation utility.
- **`KFold` is reimplemented, not wrapped**, so the crate has zero required
  dependency on `smartcore`. It is functionally equivalent to
  `smartcore::model_selection::KFold`.

## MSRV & license

MSRV 1.74. Licensed under [MIT](LICENSE).

[`smartcore::model_selection`]: https://docs.rs/smartcore
[`rayon`]: https://crates.io/crates/rayon