optimizer 1.0.1

Bayesian and population-based optimization library with an Optuna-like API for hyperparameter tuning and black-box optimization
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

//! Tree-Parzen Estimator (TPE) sampler family for Bayesian optimization.
//!
//! TPE is a sequential model-based optimization algorithm that models P(x|y) instead
//! of P(y|x). It splits completed trials into "good" (below the gamma quantile) and
//! "bad" groups, fits a kernel density estimator (KDE) to each, and proposes new
//! points by maximizing the l(x)/g(x) ratio — an approximation of Expected Improvement.
//!
//! # Samplers
//!
//! | Sampler | Models parameters | Best for |
//! |---------|-------------------|----------|
//! | [`TpeSampler`] | Independently | General-purpose single-objective optimization |
//! | [`MultivariateTpeSampler`] | Jointly | Problems with correlated parameters |
//!
//! # Gamma strategies
//!
//! The gamma quantile controls how many trials are considered "good". This module
//! provides four built-in strategies via the [`GammaStrategy`] trait:
//!
//! | Strategy | Formula | Default |
//! |----------|---------|---------|
//! | [`FixedGamma`] | Constant value | gamma = 0.25 |
//! | [`LinearGamma`] | Linear ramp from min to max | 0.10 → 0.25 over 100 trials |
//! | [`SqrtGamma`] | 1/√n decay (Optuna-style) | factor = 1.0, max = 0.25 |
//! | [`HyperoptGamma`] | (base+1)/n (Hyperopt-style) | base = 24, max = 0.25 |
//!
//! You can also implement [`GammaStrategy`] for a custom splitting rule.
//!
//! # Search-space utilities
//!
//! The [`search_space`] submodule provides [`IntersectionSearchSpace`] for computing
//! the common parameter set across trials, and [`GroupDecomposedSearchSpace`] for
//! splitting parameters into independent groups based on co-occurrence.
//!
//! # Examples
//!
//! Basic TPE with default settings:
//!
//! ```
//! use optimizer::sampler::tpe::TpeSampler;
//! use optimizer::{Direction, Study};
//!
//! let study: Study<f64> = Study::with_sampler(Direction::Minimize, TpeSampler::new());
//! ```
//!
//! Multivariate TPE for correlated parameters:
//!
//! ```
//! use optimizer::sampler::tpe::MultivariateTpeSampler;
//! use optimizer::{Direction, Study};
//!
//! let sampler = MultivariateTpeSampler::builder()
//!     .gamma(0.15)
//!     .n_startup_trials(20)
//!     .group(true)
//!     .seed(42)
//!     .build()
//!     .unwrap();
//! let study: Study<f64> = Study::with_sampler(Direction::Minimize, sampler);
//! ```

pub(crate) mod common;
mod gamma;
mod multivariate;
mod sampler;
pub mod search_space;

pub use gamma::{FixedGamma, GammaStrategy, HyperoptGamma, LinearGamma, SqrtGamma};
pub use multivariate::{
    ConstantLiarStrategy, MultivariateTpeSampler, MultivariateTpeSamplerBuilder,
};
pub use sampler::{TpeSampler, TpeSamplerBuilder};
pub use search_space::{GroupDecomposedSearchSpace, IntersectionSearchSpace};