Struct LBFGS 

pub struct LBFGS { /* private fields */ }

Limited-memory BFGS (L-BFGS) optimizer.

L-BFGS is a quasi-Newton method that approximates the inverse Hessian using a limited history of gradient information. It’s efficient for large-scale optimization problems where storing the full Hessian is infeasible.

Algorithm

1. Compute gradient g_k = ∇f(x_k)
2. Compute search direction d_k using two-loop recursion (approximates H^(-1) * g_k)
3. Find step size α_k via line search (Wolfe conditions)
4. Update: x_{k+1} = x_k - α_k * d_k
5. Store gradient and position differences for next iteration

Parameters

• max_iter: Maximum number of iterations
• tol: Convergence tolerance (gradient norm)
• m: History size (typically 5-20, tradeoff between memory and convergence)

Precision

This type is f32 and NON-generic — deliberately, because adding a generic or default type parameter would break downstream use sites. For f64 (for example a softmax-NLL objective whose gradient norm reaches f32 epsilon near the optimum) use LbfgsF64.

Example

use aprender::optim::{LBFGS, Optimizer};
use aprender::primitives::Vector;

let mut optimizer = LBFGS::new(100, 1e-5, 10);

// Define Rosenbrock function and its gradient
let f = |x: &Vector<f32>| {
    let a = x[0];
    let b = x[1];
    (1.0 - a).powi(2) + 100.0 * (b - a * a).powi(2)
};

let grad = |x: &Vector<f32>| {
    let a = x[0];
    let b = x[1];
    Vector::from_slice(&[
        -2.0 * (1.0 - a) - 400.0 * a * (b - a * a),
        200.0 * (b - a * a),
    ])
};

let x0 = Vector::from_slice(&[0.0, 0.0]);
let result = optimizer.minimize(f, grad, x0);

// Should converge to (1, 1)
assert_eq!(result.status, aprender::optim::ConvergenceStatus::Converged);

Implementations

impl LBFGS

pub fn new(max_iter: usize, tol: f32, m: usize) -> LBFGS

Creates a new L-BFGS optimizer.

Arguments

• max_iter - Maximum number of iterations (typical: 100-1000)
• tol - Convergence tolerance for gradient norm (typical: 1e-5)
• m - History size (typical: 5-20)

Example

use aprender::optim::LBFGS;

let optimizer = LBFGS::new(100, 1e-5, 10);

Trait Implementations

impl Clone for LBFGS

fn clone(&self) -> LBFGS

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LBFGS

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Optimizer for LBFGS

fn step(&mut self, _params: &mut Vector<f32>, _gradients: &Vector<f32>)

Stochastic update (mini-batch mode) - for SGD, Adam, RMSprop.

fn minimize<F, G>( &mut self, objective: F, gradient: G, x0: Vector<f32>, ) -> OptimizationResult

where F: Fn(&Vector<f32>) -> f32, G: Fn(&Vector<f32>) -> Vector<f32>,

Batch optimization (deterministic mode) - for L-BFGS, CG, Damped Newton.

fn reset(&mut self)

Resets the optimizer state (momentum, history, etc.).