Struct DampedNewton 

pub struct DampedNewton { /* private fields */ }

Damped Newton optimizer with finite-difference Hessian approximation.

Newton’s method uses second-order information (Hessian) to find the minimum by solving the linear system: H * d = -g, where H is the Hessian and g is the gradient. The damping factor and line search ensure global convergence.

Algorithm

1. Compute gradient g = ∇f(x)
2. Approximate Hessian H using finite differences
3. Solve H * d = -g using Cholesky decomposition
4. If Hessian not positive definite, fall back to steepest descent
5. Line search along d to find step size α
6. Update: x_{k+1} = x_k + α * d_k

Parameters

• max_iter: Maximum number of iterations
• tol: Convergence tolerance (gradient norm)
• epsilon: Finite difference step size for Hessian approximation (default: 1e-5)

Example

use aprender::optim::{DampedNewton, Optimizer};
use aprender::primitives::Vector;

let mut optimizer = DampedNewton::new(100, 1e-5);

// Minimize quadratic function f(x,y) = x^2 + 2y^2
let f = |x: &Vector<f32>| x[0] * x[0] + 2.0 * x[1] * x[1];
let grad = |x: &Vector<f32>| Vector::from_slice(&[2.0 * x[0], 4.0 * x[1]]);

let x0 = Vector::from_slice(&[5.0, 3.0]);
let result = optimizer.minimize(f, grad, x0);

Implementations

impl DampedNewton

pub fn new(max_iter: usize, tol: f32) -> Self

Creates a new Damped Newton optimizer.

Arguments

• max_iter - Maximum number of iterations (typical: 100-1000)
• tol - Convergence tolerance for gradient norm (typical: 1e-5)

Example

use aprender::optim::DampedNewton;

let optimizer = DampedNewton::new(100, 1e-5);

pub fn with_epsilon(self, epsilon: f32) -> Self

Sets the finite difference epsilon for Hessian approximation.

Arguments

• epsilon - Step size for finite differences (typical: 1e-5 to 1e-8)

Example

use aprender::optim::DampedNewton;

let optimizer = DampedNewton::new(100, 1e-5).with_epsilon(1e-6);

Trait Implementations

impl Clone for DampedNewton

fn clone(&self) -> DampedNewton

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DampedNewton

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for DampedNewton

fn step(&mut self, _params: &mut Vector<f32>, _gradients: &Vector<f32>)

Stochastic update (mini-batch mode) - for SGD, Adam, RMSprop.

fn minimize<F, G>( &mut self, objective: F, gradient: G, x0: Vector<f32>, ) -> OptimizationResult

where F: Fn(&Vector<f32>) -> f32, G: Fn(&Vector<f32>) -> Vector<f32>,

Batch optimization (deterministic mode) - for L-BFGS, CG, Damped Newton.

fn reset(&mut self)

Resets the optimizer state (momentum, history, etc.).