Struct AdamW 

pub struct AdamW { /* private fields */ }

AdamW optimizer (Loshchilov & Hutter, 2019).

Like Adam but with decoupled weight decay, which is more effective for regularization.

The key difference from Adam:

param = param - lr * weight_decay * param  // Decoupled weight decay
param = param - lr * m̂_t / (√v̂_t + ε)      // Then Adam update

Implementations

impl AdamW

pub fn new(params: Vec<&mut Tensor>, lr: f32) -> Self

Create a new AdamW optimizer.

Default: β₁=0.9, β₂=0.999, ε=1e-8, weight_decay=0.01

pub fn betas(self, beta1: f32, beta2: f32) -> Self

pub fn eps(self, eps: f32) -> Self

pub fn weight_decay(self, wd: f32) -> Self

pub fn step_with_params(&mut self, params: &mut [&mut Tensor])

Trait Implementations

impl Debug for AdamW

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for AdamW

fn step(&mut self)

Perform a single optimization step using computed gradients.

fn zero_grad(&mut self)

Zero all parameter gradients.

fn lr(&self) -> f32

Get current learning rate.

fn set_lr(&mut self, lr: f32)

Set learning rate (for schedulers).