Module selection 

Automated model selection based on multiple metrics

This module provides utilities for automatically selecting the best model from a set of candidates based on multiple evaluation metrics.

Features

• Multi-metric evaluation: Combine multiple metrics with custom weights
• Flexible scoring: Support different aggregation strategies
• Pareto optimal selection: Find models that are not dominated by others
• Cross-validation integration: Work with CV results for robust selection
• Custom criteria: Define custom selection criteria

Examples

Basic Model Selection

use scirs2_metrics::selection::{ModelSelector, SelectionCriteria};
use std::collections::HashMap;

// Define models and their metric scores
let mut modelscores = HashMap::new();
modelscores.insert("model_a".to_string(), vec![("accuracy", 0.85), ("precision", 0.82)]);
modelscores.insert("model_b".to_string(), vec![("accuracy", 0.80), ("precision", 0.90)]);
modelscores.insert("model_c".to_string(), vec![("accuracy", 0.88), ("precision", 0.85)]);

// Create selector with weighted criteria
let mut selector = ModelSelector::new();
selector.add_metric("accuracy", 0.6, true)  // 60% weight, higher is better
        .add_metric("precision", 0.4, true); // 40% weight, higher is better

// Select best model
let best_model = selector.select_best(&modelscores).unwrap();
println!("Best model: {}", best_model);

Pareto Optimal Selection

use scirs2_metrics::selection::ModelSelector;
use std::collections::HashMap;

let mut modelscores = HashMap::new();
modelscores.insert("model_a".to_string(), vec![("accuracy", 0.85), ("speed", 100.0)]);
modelscores.insert("model_b".to_string(), vec![("accuracy", 0.80), ("speed", 200.0)]);
modelscores.insert("model_c".to_string(), vec![("accuracy", 0.90), ("speed", 50.0)]);

let mut selector = ModelSelector::new();
selector
    .add_metric("accuracy", 1.0, true)   // higher is better
    .add_metric("speed", 1.0, true);     // higher is better (faster inference)

let pareto_optimal = selector.find_pareto_optimal(&modelscores);
println!("Pareto optimal models: {:?}", pareto_optimal);

Structs

MetricCriterion

Represents a metric with its weight and optimization direction

ModelSelectionBuilder

Builder for creating complex model selection scenarios

ModelSelector

Main model selector that evaluates and ranks models

SelectionCriteria

Model selection criteria configuration

SelectionResult

Represents the result of model selection with detailed information

Enums

AggregationStrategy

Aggregation strategies for combining multiple metrics