Module gradient 

First-order optimizers: learning-rate methods (gradient descent, SGD, Adam, RMSProp, AdaGrad) and the Fletcher–Reeves conjugate gradient.

Each optimizer drives an Objective toward a minimum and returns an OptimizeResult. The learning-rate methods stop when the gradient norm falls below a tolerance; conjugate gradient additionally restarts every n steps.

Re-exports

pub use adagrad::adagrad;

pub use adam::adam;

pub use conjugate_gradient::conjugate_gradient;

pub use gradient_descent::gradient_descent;

pub use rmsprop::rmsprop;

pub use sgd::sgd;

Modules

adagrad

AdaGrad optimizer (Duchi, Hazan & Singer 2011) — per-coordinate adaptive rates.

adam

Adam optimizer (Kingma & Ba 2014) — adaptive moment estimation.

conjugate_gradient

Nonlinear conjugate gradient (Fletcher–Reeves) with backtracking line search.

gradient_descent

Vanilla (batch) gradient descent — the worked pattern every learning-rate optimizer follows.

rmsprop

RMSProp optimizer (Tieleman & Hinton 2012) — root-mean-square gradient scaling.

sgd

Stochastic gradient descent with seeded coordinate-subsampled steps.