Crate optimization [−] [src]

Collection of various optimization algorithms and strategies.

Building Blocks

Each central primitive is specified by a trait:

Function - Specifies a function that can be minimized
Derivative1 - Extends a Function by the analytical first derivative
NumericalDifferentiation - Provides numerical differentiation for arbitrary Functions
Minimizer - A minimization algorithm
Evaluation - A function evaluation f(x) = y that is returned by a Minimizer

Currently, the following algorithms are implemented. This list is not final and being expanded over time.

GradientDescent - Iterative gradient descent minimization, supporting various line search methods:
- FixedStepWidth - No line search is performed, but a fixed step width is used
- ExactLineSearch - Exhaustive line search over a set of step widths
- ArmijoLineSearch - Backtracking line search using the Armijo rule as stopping criterion

Common optimization problems for testing purposes.

ArmijoLineSearch	Backtracking line search evaluating the Armijo rule at each step width.
ExactLineSearch	Brute-force line search minimizing the objective function over a set of step width candidates, also known as exact line search.
FixedStepWidth	Uses a fixed step width `γ` in each iteration instead of performing an actual line search.
GradientDescent	A simple Gradient Descent optimizer.
NumericalDifferentiation	Wraps a function for which to provide numeric differentiation.

Derivative1	Defines an objective function `f` that is able to compute the first derivative `f'(x)` analytically.
Evaluation	Captures the essence of a function evaluation.
Function	Defines an objective function `f` that is subject to minimization.
LineSearch	Define a line search method, i.e., choosing an appropriate step width.
Minimizer	Defines an optimizer that is able to minimize a given objective function `F`.