Crate tpe

This crate provides a hyperparameter optimization algorithm using TPE (Tree-structured Parzen Estimator).

Examples

An example optimizing a simple quadratic function which has one numerical and one categorical parameters.

use rand::SeedableRng as _;

let choices = [1, 10, 100];
let mut optim0 =
    tpe::TpeOptimizer::new(tpe::parzen_estimator(), tpe::range(-5.0, 5.0)?);
let mut optim1 =
    tpe::TpeOptimizer::new(tpe::histogram_estimator(), tpe::categorical_range(choices.len())?);

fn objective(x: f64, y: i32) -> f64 {
    x.powi(2) + y as f64
}

let mut best_value = std::f64::INFINITY;
let mut rng = rand::rngs::StdRng::from_seed(Default::default());
for _ in 0..100 {
   let x = optim0.ask(&mut rng)?;
   let y = optim1.ask(&mut rng)?;

   let v = objective(x, choices[y as usize]);
   optim0.tell(x, v)?;
   optim1.tell(y, v)?;
   best_value = best_value.min(v);
}

assert_eq!(best_value, 1.000054276671888);

References

Please refer to the following papers about the details of TPE:

• Algorithms for Hyper-Parameter Optimization
• Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures

Modules

density_estimation

Probability density function estimation.

range

Parameter range.

Structs

TpeOptimizer

Optimizer using TPE.

TpeOptimizerBuilder

Builder of TpeOptimizer.

Enums

BuildError

Possible errors during TpeOptimizerBuilder::build.

TellError

Possible errors during TpeOptimizer::tell.

Functions

categorical_range

Creates a Range for a categorical parameter.

histogram_estimator

Creates a DefaultEstimatorBuilder to build HistogramEstimator (for numerical parameter).

parzen_estimator

Creates a DefaultEstimatorBuilder to build ParzenEstimator (for categorical parameter).

range

Creates a Range instance.