Struct dfdx::optim::RMSprop

pub struct RMSprop<M, E: Dtype, D: DeviceStorage> {
    pub cfg: RMSpropConfig<E>,
    /* private fields */
}

RMSprop As described in Hinton, 2012.

This implementation is based off of RMSprop from pytorch-image-models because the pytorch implementation has some issues.

The main difference between the pytorch implementation is that RMSpropConfig::eps is added inside of the sqrt() operation.

The lr_in_momentum option is not provided because it didn’t seem to make a difference in testing.

Example Usage

Constructing using new:

let rmsprop: RMSprop<Model, f32, Cpu> = RMSprop::new(&model, RMSpropConfig {
    lr: 1e-3,
    alpha: 0.5,
    eps: 1e-8,
    momentum: Some(0.5),
    centered: false,
    weight_decay: Some(WeightDecay::Decoupled(1e-1)),
});

Fields

cfg: RMSpropConfig<E>

Hyperparameter configuration

Implementations

impl<M, E: Dtype, D: DeviceStorage> RMSprop<M, E, D>

pub fn new(_model: &M, cfg: RMSpropConfig<E>) -> Self

Constructs using hyperparameters from cfg.

Trait Implementations

impl<M: Debug, E: Debug + Dtype, D: Debug + DeviceStorage> Debug for RMSprop<M, E, D>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<M: TensorCollection<E, D>, D: Device<E> + OneFillStorage<E>, E: Dtype> Optimizer<M, D, E> for RMSprop<M, E, D>

fn update( &mut self, module: &mut M, gradients: &Gradients<E, D> ) -> Result<(), OptimizerUpdateError<D>>

Updates all of module’s parameters using gradients.