[−][src]Crate optimization
Collection of various optimization algorithms and strategies.
Building Blocks
Each central primitive is specified by a trait:
Function
- Specifies a function that can be minimizedFunction1
- Extends aFunction
by its first derivativeSummation
- Represents a summation of functions, exploited, e.g., by SGDSummation1
- Analogous toFunction
andFunction1
but forSummation
Minimizer
- A minimization algorithmEvaluation
- A function evaluationf(x) = y
that is returned by aMinimizer
Func
- A new-type wrapper for theFunction
traitNumericalDifferentiation
- Provides numerical differentiation for arbitraryFunction
s
Algorithms
Currently, the following algorithms are implemented. This list is not final and being expanded over time.
GradientDescent
- Iterative gradient descent minimization, supporting various line search methods:FixedStepWidth
- No line search is performed, but a fixed step width is usedExactLineSearch
- Exhaustive line search over a set of step widthsArmijoLineSearch
- Backtracking line search using the Armijo rule as stopping criterion
StochasticGradientDescent
- Iterative stochastic gradient descenent minimazation, currently using a fixed step width
Modules
problems | Common optimization problems for testing purposes. |
Structs
ArmijoLineSearch | Backtracking line search evaluating the Armijo rule at each step width. |
ExactLineSearch | Brute-force line search minimizing the objective function over a set of step width candidates, also known as exact line search. |
FixedStepWidth | Uses a fixed step width |
Func | New-type to support optimization of arbitrary functions without requiring to implement a trait. |
GradientDescent | A simple Gradient Descent optimizer. |
NumericalDifferentiation | Wraps a function for which to provide numeric differentiation. |
StochasticGradientDescent | Provides stochastic Gradient Descent optimization. |
Traits
Evaluation | Captures the essence of a function evaluation. |
Function | Defines an objective function |
Function1 | Defines an objective function |
LineSearch | Define a line search method, i.e., choosing an appropriate step width. |
Minimizer | Defines an optimizer that is able to minimize a given objective function |
Summation | Defines a summation of individual functions, i.e., f(x) = ∑ᵢ fᵢ(x). |
Summation1 | Defines a summation of individual functions |