Struct Adam 

pub struct Adam { /* private fields */ }

Adam optimiser

This includes AdamW via use of decoupled weight decay

Described in Adam: A Method for Stochastic Optimization and Decoupled Weight Decay Regularization

The AMSGrad variant is also implemented, described in On the Convergence of Adam and Beyond

Implementations

impl Adam

pub fn into_inner(self) -> Vec<Var>

Return the vars being optimised

pub fn set_betas(&mut self, beta_1: f64, beta_2: f64)

set the betas

this can be combined with set_lr for LR and momentum decay scheduling

Trait Implementations

impl Debug for Adam

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl OptimParams for Adam

fn set_params(&mut self, config: Self::Config)

Set the parameters for the optimiser

Warning

As the AMSGrad variant requires having tracked an additional tensor this variable cannot be changed once set initally on creation of the optimiser.

fn params(&self) -> &Self::Config

get the current parameters of the Optimiser

impl Optimizer for Adam

type Config = ParamsAdam

fn new(vars: Vec<Var>, params: ParamsAdam) -> Result<Self>

fn learning_rate(&self) -> f64

fn step(&mut self, grads: &GradStore) -> Result<()>

fn set_learning_rate(&mut self, lr: f64)

fn empty(config: Self::Config) -> Result<Self, Error>

fn backward_step(&mut self, loss: &Tensor) -> Result<(), Error>

fn from_slice(vars: &[&Var], config: Self::Config) -> Result<Self, Error>