Struct GradientDescent 

pub struct GradientDescent<L, V, F = f64> { /* private fields */ }

Steepest-descent solver: step in the direction of −∇f(x) with a pluggable line search and optional heavy-ball momentum.

The line search type parameter L is the strategy (e.g. Constant, Backtracking, Wolfe). Use GradientDescent::new for a fixed step or GradientDescent::with_line_search to pick a strategy explicitly.

Momentum

with_momentum adds a heavy-ball velocity term (Polyak 1964). With momentum coefficient β and the per-step length αₖ chosen by the line search, the update becomes

vₖ₊₁ = β · vₖ − αₖ · ∇f(xₖ)
xₖ₊₁ = xₖ + vₖ₊₁

starting from v₀ = 0. β = 0 (the default) is exactly plain steepest descent; β ∈ (0, 1) carries momentum, which cancels the oscillating component of the gradient across a narrow valley while accumulating speed along the valley floor. With a Constant step this is the classical heavy-ball method — well-behaved on the curved, ill-conditioned Rosenbrock valley where plain steepest descent zig-zags. A too-large effective step (roughly α / (1 − β) along consistent directions) diverges, so reduce α when adding momentum.

Backends

Backend-generic — works with any V implementing ScaledAdd<F> + NegInPlace + ScaleInPlace<F> + Clone. With the default F = f64 that covers Vec<f64>, nalgebra::DVector<f64> (feature nalgebra), ndarray::Array1<f64> (feature ndarray), and faer::Col<f64> (feature faer).

References

Polyak, B. T. (1964). “Some methods of speeding up the convergence of iteration methods.” USSR Computational Mathematics and Mathematical Physics, 4(5), 1–17. doi:10.1016/0041-5553(64)90137-5.

Examples

Minimize the 2-D sphere f(x) = x₀² + x₁² from (1, 1):

use basin::{BasicState, CostFunction, Executor, Gradient, GradientDescent, GradientTolerance};

struct Sphere;
impl CostFunction for Sphere {
    type Param = Vec<f64>;
    type Output = f64;
    type Error = std::convert::Infallible;
    fn cost(&self, x: &Vec<f64>) -> Result<f64, Self::Error> {
        Ok(x.iter().map(|xi| xi * xi).sum())
    }
}
impl Gradient for Sphere {
    type Gradient = Vec<f64>;
    fn gradient(&self, x: &Vec<f64>) -> Result<Vec<f64>, Self::Error> {
        Ok(x.iter().map(|xi| 2.0 * xi).collect())
    }
}

let result = Executor::new(Sphere, GradientDescent::new(0.1), BasicState::new(vec![1.0, 1.0]))
    .max_iter(1_000)
    .terminate_on(GradientTolerance(1e-8))
    .run()
    .unwrap();
assert!(result.cost() < 1e-12);

Implementations

impl<V, F: Scalar> GradientDescent<Constant<F>, V, F>

pub fn new(alpha: F) -> Self

Gradient descent with a fixed step size alpha. Equivalent to with_line_search(Constant(alpha)).

impl<L, V, F: Scalar> GradientDescent<L, V, F>

pub fn with_line_search(line_search: L) -> Self

Gradient descent with an explicit line-search strategy (e.g. Backtracking, Wolfe).

pub fn with_momentum(self, beta: F) -> Self

Enable heavy-ball momentum with coefficient beta (Polyak 1964). beta = 0.0 is plain steepest descent; beta in (0, 1) (commonly 0.9) adds momentum. See the type docs for the update rule and stability caveat.

Trait Implementations

impl<L, V, F> InitialState<V> for GradientDescent<L, V, F>

where F: Scalar, V: Clone,

Lets GradientDescent serve as the inner of a composed solver (e.g. BarrierMethod / AugmentedLagrangianMethod), seeding a fresh BasicState at the warm-start point.

type State = BasicState<V, F>

State shape this solver iterates against.

fn seed(&self, x: &V) -> BasicState<V, F>

Build a fresh state seeded at x using the solver’s natural default scale.

impl<P, V, F, L> Solver<P, BasicState<V, F>> for GradientDescent<L, V, F>

where F: Scalar, P: CostFunction<Param = V, Output = F> + Gradient<Gradient = V>, V: ScaledAdd<F> + NegInPlace + ScaleInPlace<F> + Clone, L: LineSearch<P, V, F, Error = P::Error>,

type Error = <P as CostFunction>::Error

Hard-abort error type — mirrors the underlying problem’s type Error. See the trait docs.

fn init( &mut self, problem: &mut Problem<P>, state: BasicState<V, F>, ) -> Result<BasicState<V, F>, Self::Error>

One-time setup before the iteration loop.

fn next_iter( &mut self, problem: &mut Problem<P>, state: BasicState<V, F>, ) -> Result<(BasicState<V, F>, Option<TerminationReason>), Self::Error>

Advance one iteration.

fn terminate(&self, _state: &S) -> Option<TerminationReason>

Optional pre-iteration solver-specific termination test.

impl<L, V, F> WarmStart<V> for GradientDescent<L, V, F>

where F: Scalar, V: Clone,