Module second_order 

Second-order stochastic optimization algorithms.

This module provides second-order quasi-Newton methods suitable for both deterministic and stochastic optimization:

• L-BFGS-B (lbfgsb): Limited-memory BFGS with box constraints, featuring strong Wolfe line search and Cauchy-point computation.
• SR1 (sr1): Symmetric rank-1 quasi-Newton with trust-region globalization and limited-memory compact representation.
• S-L-BFGS (slbfgs): Stochastic L-BFGS combining mini-batch gradients, curvature pairs from large batches, and optional SVRG variance reduction.

References

• Byrd et al. (1995). “A limited memory algorithm for bound constrained optimization.” SIAM J. Sci. Comput.
• Byrd, Khalfan & Schnabel (1994). “Analysis of a symmetric rank-one trust region method.” SIAM J. Optim.
• Moritz, Nishihara & Jordan (2016). “A linearly-convergent stochastic L-BFGS algorithm.” AISTATS.

Re-exports

pub use lbfgsb::cauchy_point;

pub use lbfgsb::hv_product;

pub use lbfgsb::project;

pub use lbfgsb::projected_grad_norm;

pub use lbfgsb::wolfe_line_search;

pub use lbfgsb::LbfgsBOptimizer;

pub use slbfgs::Lcg;

pub use slbfgs::SlbfgsOptimizer;

pub use sr1::lsr1_hv_product;

pub use sr1::sr1_update_dense;

pub use sr1::trust_region_step;

pub use sr1::Sr1Optimizer;

pub use types::HessianApprox;

pub use types::LbfgsBConfig;

pub use types::OptResult;

pub use types::SlbfgsConfig;

pub use types::Sr1Config;

Modules

lbfgsb

L-BFGS-B: Limited-memory BFGS with box constraints (Byrd et al. 1995).

slbfgs

Stochastic L-BFGS (S-L-BFGS) with SVRG-style variance reduction.

sr1

SR1 (Symmetric Rank-1) quasi-Newton optimizer with trust-region globalization.

types

Types and configuration structs for second-order stochastic optimization.