Struct dfdx::optim::RMSprop

pub struct RMSprop {
    pub cfg: RMSpropConfig,
    /* private fields */
}

RMSprop As described in Hinton, 2012.

This implementation is based off of RMSprop from pytorch-image-models.

The pytorch implementation has some issues: https://github.com/pytorch/pytorch/issues/23796.

Note: we don’t provide lr_in_momentum option because it didn’t seem to make a difference in testing.

Example Usage:

let mut t = Tensor0D::ones();
let mut opt: RMSprop = Default::default();
let gradients = t.trace().backward();
opt.update(&mut t, gradients);

Changing default parmeters:

let rmsprop = RMSprop::new(RMSpropConfig {
    lr: 1e-3,
    alpha: 0.5,
    ..Default::default()
});

Fields

cfg: RMSpropConfig

Configuration like learning rate and momentum.

Implementations

impl RMSprop

pub fn new(cfg: RMSpropConfig) -> Self

Constructs a new RMSprop using cfg RMSpropConfig.

Trait Implementations

impl Debug for RMSprop

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RMSprop

fn default() -> Self

See RMSpropConfig::default()

impl GradientProvider for RMSprop

fn gradient<P>(&mut self, p: &P) -> Box<P::Array> where
    P: HasUniqueId + HasArrayType<Dtype = f32> + HasDevice, 

Retrieves the data associated with p if there is any. This can modify self, for instance if velocities are calculated based on the associated data!

impl Optimizer for RMSprop

fn update<M: CanUpdateWithGradients>(
    &mut self,
    module: &mut M,
    gradients: Gradients
)