tpe 0.3.0

A library that implements TPE, a hyperparameter optimization algorithm
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

tpe
===

[![tpe](https://img.shields.io/crates/v/tpe.svg)](https://crates.io/crates/tpe)
[![Documentation](https://docs.rs/tpe/badge.svg)](https://docs.rs/tpe)
[![Actions Status](https://github.com/sile/tpe/workflows/CI/badge.svg)](https://github.com/sile/tpe/actions)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

This crate provides a hyperparameter optimization algorithm using TPE (Tree-structured Parzen Estimator).


Examples
--------

### Minimize the result of a quadratic function

An example optimizing a simple quadratic function which has one numerical and one categorical parameter.
```rust
use rand::SeedableRng as _;

let choices = [1, 10, 100];
let mut optim0 =
    tpe::TpeOptimizer::new(tpe::parzen_estimator(), tpe::range(-5.0, 5.0)?);
let mut optim1 =
    tpe::TpeOptimizer::new(tpe::histogram_estimator(), tpe::categorical_range(choices.len())?);

fn objective(x: f64, y: i32) -> f64 {
    x.powi(2) + y as f64
}

let mut best_value = f64::INFINITY;
let mut rng = rand::rngs::StdRng::from_seed(Default::default());
for _ in 0..100 {
   let x = optim0.ask(&mut rng)?;
   let y = optim1.ask(&mut rng)?;

   let v = objective(x, choices[y as usize]);
   optim0.tell(x, v)?;
   optim1.tell(y, v)?;
   best_value = best_value.min(v);
}

assert_eq!(best_value, 1.0001089491396404);
```


References
----------

Please refer to the following papers about the details of TPE:

- [Algorithms for Hyper-Parameter Optimization](https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf)
- [Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures](http://proceedings.mlr.press/v28/bergstra13.pdf)