Module optimizers 

Stateful first-order stochastic optimizers with learning rate schedules.

Provides clean, struct-based implementations of:

• SGD — vanilla, momentum, Nesterov momentum, weight-decay
• Adam — adaptive moment estimation
• AdaGrad — cumulative gradient-square scaling
• RMSprop — exponential moving-average second moment
• AdamW — Adam with decoupled weight decay
• SVRG — Stochastic Variance Reduced Gradient
• LrSchedule — a rich enum of learning rate schedules

Usage pattern

use scirs2_optimize::stochastic::optimizers::{Sgd, LrSchedule};

let mut opt = Sgd::new(0.01, 0.9);
let mut params = vec![1.0_f64, -2.0];
for _ in 0..100 {
    let grad = params.iter().map(|&p| 2.0 * p).collect::<Vec<_>>();
    opt.step(&mut params, &grad).expect("valid input");
}

Structs

AdaGrad

AdaGrad optimizer (Duchi et al. 2011).

Adam

Adam optimizer (Kingma & Ba 2015).

AdamW

AdamW optimizer (Loshchilov & Hutter 2017) — Adam with decoupled weight decay.

RmsProp

RMSprop optimizer (Hinton 2012).

Sgd

Stochastic Gradient Descent with optional momentum, Nesterov lookahead, and L2 weight decay.

Svrg

Stochastic Variance Reduced Gradient (SVRG; Johnson & Zhang 2013).

Enums

LrSchedule

Learning rate schedule variants.