Struct dfdx::optim::Sgd

pub struct Sgd<M, E: Dtype, D: Storage<E>> {
    pub cfg: SgdConfig,
    /* private fields */
}

Implementation of Stochastic Gradient Descent. Based on pytorch’s implementation

Nesterov Momentum is implemented as described in On the importance of initialization and momentum in deep learning.

Weight decay is implemented as described in Decoupled Weight Decay Regularization Both L2 weight_decay and decoupled weight_decay are available.

Example Usage

let mut opt: Sgd<Model, f32, Cpu> = Sgd::new(&model, SgdConfig {
    lr: 1e-3,
    momentum: Some(Momentum::Classic(0.5)),
    weight_decay: Some(WeightDecay::L2(0.01)),
});

See module level documentation at crate::optim for examples of how to actually use an optimizer.

Fields

cfg: SgdConfig

Hyperparameter configuration

Implementations

impl<M, E: Dtype, D: Storage<E>> Sgd<M, E, D>

pub fn new(_model: &M, cfg: SgdConfig) -> Self

Constructs using hyperparameters from cfg

Trait Implementations

impl<M: Clone, E: Clone + Dtype, D: Clone + Storage<E>> Clone for Sgd<M, E, D>

fn clone(&self) -> Sgd<M, E, D>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<M: Debug, E: Debug + Dtype, D: Debug + Storage<E>> Debug for Sgd<M, E, D>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<M: TensorCollection<E, D>, D: Device<E>, E: Dtype> Optimizer<M, D, E> for Sgd<M, E, D>

fn update( &mut self, module: &mut M, gradients: &Gradients<E, D> ) -> Result<(), OptimizerUpdateError<D::Err>>

Updates all of module’s parameters using gradients.