Struct ConjugateGradient 

pub struct ConjugateGradient { /* private fields */ }

Nonlinear Conjugate Gradient (CG) optimizer.

CG is an iterative method for solving optimization problems that uses conjugate directions rather than steepest descent. It’s particularly effective for quadratic problems but extends to general nonlinear optimization.

Algorithm

1. Initialize with steepest descent: d_0 = -∇f(x_0)
2. Line search: find α_k minimizing f(x_k + α_k d_k)
3. Update: x_{k+1} = x_k + α_k d_k
4. Compute β_{k+1} using chosen formula (FR, PR, or HS)
5. Update direction: d_{k+1} = -∇f(x_{k+1}) + β_{k+1} d_k
6. Restart if β < 0 or every n iterations

Parameters

• max_iter: Maximum number of iterations
• tol: Convergence tolerance (gradient norm)
• beta_formula: Method for computing β (FR, PR, or HS)
• restart_interval: Restart with steepest descent every n iterations (0 = no periodic restart)

Example

use aprender::optim::{ConjugateGradient, CGBetaFormula, Optimizer};
use aprender::primitives::Vector;

let mut optimizer = ConjugateGradient::new(100, 1e-5, CGBetaFormula::PolakRibiere);

// Minimize Rosenbrock function
let f = |x: &Vector<f32>| {
    let a = x[0];
    let b = x[1];
    (1.0 - a).powi(2) + 100.0 * (b - a * a).powi(2)
};

let grad = |x: &Vector<f32>| {
    let a = x[0];
    let b = x[1];
    Vector::from_slice(&[
        -2.0 * (1.0 - a) - 400.0 * a * (b - a * a),
        200.0 * (b - a * a),
    ])
};

let x0 = Vector::from_slice(&[0.0, 0.0]);
let result = optimizer.minimize(f, grad, x0);

Implementations

impl ConjugateGradient

pub fn new(max_iter: usize, tol: f32, beta_formula: CGBetaFormula) -> Self

Creates a new Conjugate Gradient optimizer.

Arguments

• max_iter - Maximum number of iterations (typical: 100-1000)
• tol - Convergence tolerance for gradient norm (typical: 1e-5)
• beta_formula - Method for computing β (FletcherReeves, PolakRibiere, or HestenesStiefel)

Example

use aprender::optim::{ConjugateGradient, CGBetaFormula};

let optimizer = ConjugateGradient::new(100, 1e-5, CGBetaFormula::PolakRibiere);

pub fn with_restart_interval(self, interval: usize) -> Self

Sets the restart interval.

CG will restart with steepest descent every n iterations. Setting to 0 disables periodic restart (only restarts on β < 0).

Arguments

• interval - Number of iterations between restarts (typical: n, where n is problem dimension)

Example

use aprender::optim::{ConjugateGradient, CGBetaFormula};

let optimizer = ConjugateGradient::new(100, 1e-5, CGBetaFormula::PolakRibiere)
    .with_restart_interval(50);

Trait Implementations

impl Clone for ConjugateGradient

fn clone(&self) -> ConjugateGradient

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ConjugateGradient

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for ConjugateGradient

fn step(&mut self, _params: &mut Vector<f32>, _gradients: &Vector<f32>)

Stochastic update (mini-batch mode) - for SGD, Adam, RMSprop.

fn minimize<F, G>( &mut self, objective: F, gradient: G, x0: Vector<f32>, ) -> OptimizationResult

where F: Fn(&Vector<f32>) -> f32, G: Fn(&Vector<f32>) -> Vector<f32>,

Batch optimization (deterministic mode) - for L-BFGS, CG, Damped Newton.

fn reset(&mut self)

Resets the optimizer state (momentum, history, etc.).