Struct AdamW 

pub struct AdamW { /* private fields */ }

AdamW optimizer (Adam with decoupled weight decay).

Unlike standard Adam which applies L2 regularization to the gradient, AdamW applies weight decay directly to the parameters.

Update rule:

m_t = beta1 * m_{t-1} + (1 - beta1) * grad
v_t = beta2 * v_{t-1} + (1 - beta2) * grad^2
m_hat = m_t / (1 - beta1^t)
v_hat = v_t / (1 - beta2^t)
param = param - lr * (m_hat / (sqrt(v_hat) + eps) + weight_decay * param)

Implementations

impl AdamW

pub fn new(params: Vec<Parameter>, lr: f32) -> Self

Creates a new AdamW optimizer with default hyperparameters.

pub fn with_betas(params: Vec<Parameter>, lr: f32, betas: (f32, f32)) -> Self

Creates AdamW with specified betas.

pub fn with_options( params: Vec<Parameter>, lr: f32, betas: (f32, f32), eps: f32, weight_decay: f32, amsgrad: bool, ) -> Self

Creates AdamW with all options.

pub fn betas(self, betas: (f32, f32)) -> Self

Builder method to set betas.

pub fn eps(self, eps: f32) -> Self

Builder method to set epsilon.

pub fn weight_decay(self, weight_decay: f32) -> Self

Builder method to set weight decay.

pub fn amsgrad(self, amsgrad: bool) -> Self

Builder method to enable AMSGrad.

Trait Implementations

impl Optimizer for AdamW

fn step(&mut self)

Performs a single optimization step.

fn zero_grad(&mut self)

Zeros all parameter gradients.

fn get_lr(&self) -> f32

Returns the current learning rate.

fn set_lr(&mut self, lr: f32)

Sets the learning rate.

fn parameters(&self) -> &[Parameter]

Returns the parameters being optimized.

fn num_parameters(&self) -> usize

Returns the number of parameters.