Struct AugmentedLagrangian 

pub struct AugmentedLagrangian { /* private fields */ }

Augmented Lagrangian method for constrained optimization.

Solves problems with equality constraints:

minimize f(x)
subject to: h(x) = 0  (equality constraints)

Algorithm

Augmented Lagrangian: L_ρ(x, λ) = f(x) + λᵀh(x) + ½ρ‖h(x)‖²

for k = 1, 2, ..., max_iter:
    x_k = argmin L_ρ(x, λ_k)
    λ_k+1 = λ_k + ρ h(x_k)
    if ‖h(x)‖ is small: converged
    else: increase ρ

Key Features

• Penalty parameter ρ: Increased adaptively for faster convergence
• Lagrange multipliers λ: Automatically updated for equality constraints
• Flexible subproblem solver: Uses gradient descent for inner loop
• Convergence: Superlinear under regularity conditions

Applications

• Equality constraints: Linear systems, manifold optimization
• ADMM: Alternating Direction Method of Multipliers (special case)
• Consensus optimization: Distributed optimization, federated learning
• PDE-constrained optimization: Physics-informed neural networks

Example

use aprender::optim::AugmentedLagrangian;
use aprender::primitives::Vector;

// Minimize: ½(x₁-2)² + ½(x₂-3)² subject to x₁ + x₂ = 1
let objective = |x: &Vector<f32>| {
    0.5 * (x[0] - 2.0).powi(2) + 0.5 * (x[1] - 3.0).powi(2)
};

let gradient = |x: &Vector<f32>| {
    Vector::from_slice(&[x[0] - 2.0, x[1] - 3.0])
};

// Equality constraint: x₁ + x₂ - 1 = 0
let equality = |x: &Vector<f32>| Vector::from_slice(&[x[0] + x[1] - 1.0]);

let equality_jac = |_x: &Vector<f32>| {
    vec![Vector::from_slice(&[1.0, 1.0])]
};

let mut al = AugmentedLagrangian::new(100, 1e-6, 1.0);
let x0 = Vector::zeros(2);
let result = al.minimize_equality(objective, gradient, equality, equality_jac, x0);

assert!(result.constraint_violation < 1e-4);

References

• Nocedal & Wright (2006). “Numerical Optimization.” Chapter 17.
• Bertsekas (1982). “Constrained Optimization and Lagrange Multiplier Methods.”

Implementations

impl AugmentedLagrangian

pub fn new(max_iter: usize, tol: f32, initial_rho: f32) -> Self

Creates a new Augmented Lagrangian optimizer.

Arguments

• max_iter - Maximum number of outer iterations
• tol - Convergence tolerance for constraint violation
• initial_rho - Initial penalty parameter (typically 1.0-10.0)

Example

use aprender::optim::AugmentedLagrangian;

let optimizer = AugmentedLagrangian::new(100, 1e-6, 1.0);

pub fn with_rho_increase(self, factor: f32) -> Self

Sets penalty increase factor.

Arguments

• factor - Penalty increase factor (> 1, typically 2.0-10.0)

Example

use aprender::optim::AugmentedLagrangian;

let optimizer = AugmentedLagrangian::new(100, 1e-6, 1.0)
    .with_rho_increase(5.0);

pub fn minimize_equality<F, G, H, J>( &mut self, objective: F, gradient: G, equality: H, equality_jac: J, x0: Vector<f32>, ) -> OptimizationResult

where F: Fn(&Vector<f32>) -> f32, G: Fn(&Vector<f32>) -> Vector<f32>, H: Fn(&Vector<f32>) -> Vector<f32>, J: Fn(&Vector<f32>) -> Vec<Vector<f32>>,

Minimizes objective subject to equality constraints.

Solves: minimize f(x) subject to h(x) = 0

Arguments

• objective - Objective function f(x)
• gradient - Gradient ∇f(x)
• equality - Equality constraints h(x) = 0 (returns vector)
• equality_jac - Jacobian of equality constraints ∇h(x)
• x0 - Initial point

Returns

Optimization result with constraint satisfaction metrics

Trait Implementations

impl Clone for AugmentedLagrangian

fn clone(&self) -> AugmentedLagrangian

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AugmentedLagrangian

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for AugmentedLagrangian

fn step(&mut self, _params: &mut Vector<f32>, _gradients: &Vector<f32>)

Stochastic update (mini-batch mode) - for SGD, Adam, RMSprop.

fn reset(&mut self)

Resets the optimizer state (momentum, history, etc.).

fn minimize<F, G>( &mut self, _objective: F, _gradient: G, _x0: Vector<f32>, ) -> OptimizationResult

where F: Fn(&Vector<f32>) -> f32, G: Fn(&Vector<f32>) -> Vector<f32>,

Batch optimization (deterministic mode) - for L-BFGS, CG, Damped Newton.