Struct AdaBeliefOptimizer 

pub struct AdaBeliefOptimizer { /* private fields */ }

AdaBelief optimizer (NeurIPS 2020).

AdaBelief adapts the step size according to the “belief” in the gradient direction. It uses the variance of gradients (belief) to adapt the learning rate, which can achieve faster convergence and better generalization than Adam/AdamW.

Reference: Zhuang et al. “AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients” (NeurIPS 2020)

Implementations

impl AdaBeliefOptimizer

pub fn new(config: OptimizerConfig) -> Self

Create a new AdaBelief optimizer.

Trait Implementations

impl Debug for AdaBeliefOptimizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for AdaBeliefOptimizer

fn step( &mut self, parameters: &mut HashMap<String, Array<f64, Ix2>>, gradients: &HashMap<String, Array<f64, Ix2>>, ) -> TrainResult<()>

Update parameters with computed gradients.

fn zero_grad(&mut self)

Zero all gradients.

fn get_lr(&self) -> f64

Get current learning rate.

fn set_lr(&mut self, lr: f64)

Set learning rate.

fn state_dict(&self) -> HashMap<String, Vec<f64>>

Get optimizer state for checkpointing.

fn load_state_dict(&mut self, state: HashMap<String, Vec<f64>>)

Load optimizer state from checkpoint.