# [−][src]Crate optimization

Collection of various optimization algorithms and strategies.

# Building Blocks

Each central primitive is specified by a trait:

- Specifies a function that can be minimized`Function`

- Extends a`Function1`

`Function`

by its first derivative- Represents a summation of functions, exploited, e.g., by SGD`Summation`

- Analogous to`Summation1`

`Function`

and`Function1`

but for`Summation`

- A minimization algorithm`Minimizer`

- A function evaluation`Evaluation`

`f(x) = y`

that is returned by a`Minimizer`

- A new-type wrapper for the`Func`

`Function`

trait- Provides numerical differentiation for arbitrary`NumericalDifferentiation`

`Function`

s

# Algorithms

Currently, the following algorithms are implemented. This list is not final and being expanded over time.

- Iterative gradient descent minimization, supporting various line search methods:`GradientDescent`

- No line search is performed, but a fixed step width is used`FixedStepWidth`

- Exhaustive line search over a set of step widths`ExactLineSearch`

- Backtracking line search using the Armijo rule as stopping criterion`ArmijoLineSearch`

- Iterative stochastic gradient descenent minimazation, currently using a fixed step width`StochasticGradientDescent`

## Modules

problems | Common optimization problems for testing purposes. |

## Structs

ArmijoLineSearch | Backtracking line search evaluating the Armijo rule at each step width. |

ExactLineSearch | Brute-force line search minimizing the objective function over a set of step width candidates, also known as exact line search. |

FixedStepWidth | Uses a fixed step width |

Func | New-type to support optimization of arbitrary functions without requiring to implement a trait. |

GradientDescent | A simple Gradient Descent optimizer. |

NumericalDifferentiation | Wraps a function for which to provide numeric differentiation. |

StochasticGradientDescent | Provides |

## Traits

Evaluation | Captures the essence of a function evaluation. |

Function | Defines an objective function |

Function1 | Defines an objective function |

LineSearch | Define a line search method, i.e., choosing an appropriate step width. |

Minimizer | Defines an optimizer that is able to minimize a given objective function |

Summation | Defines a summation of individual functions, i.e., f(x) = ∑ᵢ fᵢ(x). |

Summation1 | Defines a summation of individual functions |