Struct dfdx::optim::Adam

pub struct Adam<M> {
    pub cfg: AdamConfig,
    /* private fields */
}

An implementation of the Adam optimizer from Adam: A Method for Stochastic Optimization

Example Usage

Constructing using default:

let mut opt: Adam<Model> = Default::default();

Changing using new

let mut opt: Adam<Model> = Adam::new(AdamConfig {
    lr: 1e-2,
    betas: [0.5, 0.25],
    eps: 1e-6,
    weight_decay: Some(WeightDecay::Decoupled(1e-2)),
});

See module level documentation at crate::optim for examples of how to actually use an optimizer.

Fields

cfg: AdamConfig

Hyperparameter configuration

Implementations

impl<M> Adam<M>

pub fn new(cfg: AdamConfig) -> Self

Constructs using hyperparameters from cfg.

Trait Implementations

impl<M: Debug> Debug for Adam<M>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<M> Default for Adam<M>

fn default() -> Self

See AdamConfig

impl<M> GradientProvider for Adam<M>

fn gradient<P>(&mut self, p: &P) -> Option<Box<P::Array>>where
    P: HasUniqueId + HasArrayType<Dtype = f32> + HasDevice + HasArrayData,

Retrieves the data associated with p if there is any. This can modify self, for instance if velocities are calculated based on the associated data!

impl<M: CanUpdateWithGradients> Optimizer<M> for Adam<M>

fn update(
    &mut self,
    module: &mut M,
    gradients: Gradients
) -> Result<(), UnusedParamsError>

Updates all of module’s parameters using gradients.