Struct RAdamOptimizer 

pub struct RAdamOptimizer { /* private fields */ }

RAdam optimizer (Rectified Adam) with variance warmup (ICLR 2020).

RAdam addresses the bad convergence problem of Adam in the early stages by rectifying the variance of the adaptive learning rate. It provides a variance warmup mechanism that stabilizes training.

Reference: Liu et al. “On the Variance of the Adaptive Learning Rate and Beyond” (ICLR 2020)

Implementations

impl RAdamOptimizer

pub fn new(config: OptimizerConfig) -> Self

Create a new RAdam optimizer.

Trait Implementations

impl Debug for RAdamOptimizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for RAdamOptimizer

fn step( &mut self, parameters: &mut HashMap<String, Array<f64, Ix2>>, gradients: &HashMap<String, Array<f64, Ix2>>, ) -> TrainResult<()>

Update parameters with computed gradients.

fn zero_grad(&mut self)

Zero all gradients.

fn get_lr(&self) -> f64

Get current learning rate.

fn set_lr(&mut self, lr: f64)

Set learning rate.

fn state_dict(&self) -> HashMap<String, Vec<f64>>

Get optimizer state for checkpointing.

fn load_state_dict(&mut self, state: HashMap<String, Vec<f64>>)

Load optimizer state from checkpoint.