opt

Dense nonlinear optimization in Rust with:

• Bfgs for first-order dense quasi-Newton optimization
• NewtonTrustRegion for Hessian-based trust-region optimization
• Arc for adaptive regularization with cubics
• FixedPoint for bounded fixed-point iteration
• automatic solver selection through Problem, SecondOrderProblem, and optimize

This crate is designed for practical nonlinear objectives, including optional simple box constraints, and is built around robustness for noisy or non-ideal functions.

This work is a rewrite of the original bfgs crate by Paul Kernfeld.

Features

• Strong Wolfe line search with practical fallback behavior for difficult first-order problems
• Dense BFGS with inverse-Hessian updates and stability safeguards
• Newton trust-region steps using supplied Hessians
• ARC with adaptive cubic regularization updates
• Fixed-point iteration with projection and step-norm termination
• Automatic solver selection for second-order objectives
• Optional simple box constraints with projected gradients
• Internal finite-difference support for cost-only and Hessian-optional objectives
• Structured error reporting with recoverable optimization failures

Usage

Add this to your Cargo.toml:

[dependencies]
opt = "0.2.0"

Example: First-Order Optimization

use opt::{
    optimize, FirstOrderObjective, FirstOrderSample, MaxIterations, Problem, Profile, Solution,
    Tolerance, ZerothOrderObjective,
};
use ndarray::{array, Array1};

struct Rosenbrock;

impl ZerothOrderObjective for Rosenbrock {
    fn eval_cost(&mut self, x: &Array1<f64>) -> Result<f64, opt::ObjectiveEvalError> {
        let a = 1.0;
        let b = 100.0;
        Ok((a - x[0]).powi(2) + b * (x[1] - x[0].powi(2)).powi(2))
    }
}

impl FirstOrderObjective for Rosenbrock {
    fn eval_grad(&mut self, x: &Array1<f64>) -> Result<FirstOrderSample, opt::ObjectiveEvalError> {
        let a = 1.0;
        let b = 100.0;
        let f = (a - x[0]).powi(2) + b * (x[1] - x[0].powi(2)).powi(2);
        Ok(FirstOrderSample {
            value: f,
            gradient: array![
                -2.0 * (a - x[0]) - 4.0 * b * (x[1] - x[0].powi(2)) * x[0],
                2.0 * b * (x[1] - x[0].powi(2)),
            ],
        })
    }
}

let x0 = array![-1.2, 1.0];

let Solution {
    final_point: x_min,
    final_value,
    final_gradient_norm: Some(grad_norm),
    iterations,
    ..
} = optimize(Problem::new(x0, Rosenbrock))
    .with_tolerance(Tolerance::new(1e-6).unwrap())
    .with_max_iterations(MaxIterations::new(100).unwrap())
    .with_profile(Profile::Robust)
    .run()
    .expect("optimization failed");

assert!((x_min[0] - 1.0).abs() < 1e-5);
assert!((x_min[1] - 1.0).abs() < 1e-5);
assert!(final_value.is_finite());
assert!(grad_norm < 1e-5);
assert!(iterations > 0);

For cost-only objectives, wrap a ZerothOrderObjective with FiniteDiffGradient.

Example: Second-Order Optimization

Use SecondOrderProblem with optimize for automatic solver selection, or construct NewtonTrustRegion and Arc directly when you want explicit control over the algorithm choice.

Workspace

This repository is a workspace with two packages:

• opt: the full solver crate
• wolfe_bfgs: a narrow BFGS-only crate layered on top of opt

Testing

Run the full workspace test suite from the repository root:

cargo test --workspace

The crate also includes comparison tests against SciPy through opt/optimization_harness.py.

License

Licensed under either of:

• Apache License, Version 2.0
• MIT license

at your option.