Struct dfdx::optim::Adam

pub struct Adam<M, E: Dtype, D: DeviceStorage> {
    pub cfg: AdamConfig<E>,
    /* private fields */
}

An implementation of the Adam optimizer from Adam: A Method for Stochastic Optimization

Example Usage

let mut opt: Adam<Model, f32, Cpu> = Adam::new(&model, AdamConfig {
    lr: 1e-2,
    betas: [0.5, 0.25],
    eps: 1e-6,
    weight_decay: Some(WeightDecay::Decoupled(1e-2)),
});

See module level documentation at crate::optim for examples of how to actually use an optimizer.

Fields

cfg: AdamConfig<E>

Hyperparameter configuration

Implementations

impl<M, E: Dtype, D: DeviceStorage> Adam<M, E, D>

pub fn new(_model: &M, cfg: AdamConfig<E>) -> Self

Constructs using hyperparameters from cfg.

Trait Implementations

impl<M: Debug, E: Debug + Dtype, D: Debug + DeviceStorage> Debug for Adam<M, E, D>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<M: TensorCollection<E, D>, D: Device<E>, E: Dtype> Optimizer<M, D, E> for Adam<M, E, D>

fn update( &mut self, module: &mut M, gradients: &Gradients<E, D> ) -> Result<(), OptimizerUpdateError<D>>

Updates all of module’s parameters using gradients.