pub struct RMSprop<M> {
pub cfg: RMSpropConfig,
/* private fields */
}
Expand description
RMSprop As described in Hinton, 2012.
This implementation is based off of RMSprop from pytorch-image-models because the pytorch implementation has some issues.
The main difference between the pytorch implementation is that RMSpropConfig::eps is added inside of the sqrt() operation.
The lr_in_momentum
option is not provided because it didn’t seem to make a difference in testing.
Example Usage
Constructing using default:
let mut opt: RMSprop<Model> = Default::default();
Constructing using new:
let rmsprop: RMSprop<Model> = RMSprop::new(RMSpropConfig {
lr: 1e-3,
alpha: 0.5,
eps: 1e-8,
momentum: Some(0.5),
centered: false,
weight_decay: Some(WeightDecay::Decoupled(1e-1)),
});
See module level documentation at crate::optim for examples of how to actually use an optimizer.
Fields
cfg: RMSpropConfig
Hyperparameter configuration
Implementations
sourceimpl<M> RMSprop<M>
impl<M> RMSprop<M>
sourcepub fn new(cfg: RMSpropConfig) -> Self
pub fn new(cfg: RMSpropConfig) -> Self
Constructs using hyperparameters from cfg
.
Trait Implementations
sourceimpl<M> GradientProvider for RMSprop<M>
impl<M> GradientProvider for RMSprop<M>
sourcefn gradient<P>(&mut self, p: &P) -> Option<Box<P::Array>>where
P: HasUniqueId + HasArrayType<Dtype = f32> + HasDevice + HasArrayData,
fn gradient<P>(&mut self, p: &P) -> Option<Box<P::Array>>where
P: HasUniqueId + HasArrayType<Dtype = f32> + HasDevice + HasArrayData,
Retrieves the data associated with
p
if there is any.
This can modify self
, for instance if velocities are calculated
based on the associated data! Read moreAuto Trait Implementations
impl<M> !RefUnwindSafe for RMSprop<M>
impl<M> !Send for RMSprop<M>
impl<M> !Sync for RMSprop<M>
impl<M> Unpin for RMSprop<M>
impl<M> !UnwindSafe for RMSprop<M>
Blanket Implementations
sourceimpl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
const: unstable · sourcefn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more