Struct dfdx::optim::SgdConfig

pub struct SgdConfig {
    pub lr: f32,
    pub momentum: Option<Momentum>,
    pub weight_decay: Option<WeightDecay>,
}

Configuration of hyperparameters for Sgd.

Using different learning rate:

SgdConfig {
    lr: 1e-1,
    momentum: None,
    weight_decay: None,
};

Using classic momentum:

SgdConfig {
    lr: 1e-2,
    momentum: Some(Momentum::Classic(0.5)),
    weight_decay: None,
};

Using nesterov momentum:

SgdConfig {
    lr: 1e-3,
    momentum: Some(Momentum::Nesterov(0.25)),
    weight_decay: None,
};

Using L2 weight decay:

SgdConfig {
    lr: 1e-3,
    momentum: None,
    weight_decay: Some(WeightDecay::L2(1e-2)),
};

Using decoupled weight decay:

SgdConfig {
    lr: 1e-3,
    momentum: None,
    weight_decay: Some(WeightDecay::Decoupled(1e-2)),
};

Fields

lr: f32

Learning rate. Defaults to 1e-2

momentum: Option<Momentum>

Optional momentum. Defaults to None.

weight_decay: Option<WeightDecay>

Optional weight decay. Defaults to None.

Trait Implementations

impl Clone for SgdConfig

fn clone(&self) -> SgdConfig

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SgdConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SgdConfig

fn default() -> Self

Returns the “default value” for a type.

impl Copy for SgdConfig