Struct zyx_optim::SGD

pub struct SGD<B: Backend> {
    pub learning_rate: f32,
    pub momentum: f32,
    pub weight_decay: f32,
    pub dampening: f32,
    pub nesterov: bool,
    pub maximize: bool,
    pub bias: Vec<Tensor<B>>,
}

Stochastic gradient descent optimizer

Fields

learning_rate: f32

learning rate (default: 0.001)

momentum: f32

momentum factor (default: 0.0)

weight_decay: f32

weight decay (L2 penalty) (default: 0.0)

dampening: f32

dampening for momentum (default: 0.0)

nesterov: bool

enables Nesterov momentum (default: false)

maximize: bool

maximize the objective with respect to the params, instead of minimizing (default: false)

bias: Vec<Tensor<B>>

stores momentum, starts empty and will be initialized on demand

Implementations

impl<B: Backend> SGD<B>

pub fn update<'a>( &mut self, parameters: impl IntoIterator<Item = &'a mut Tensor<B>>, gradients: impl IntoIterator<Item = Option<Tensor<B>>> )

where B: 'a,

Updates parameters with gradients. Number of parameters must be the same as number of gradients. Gradients can be None, those are simply skipped.

Trait Implementations

impl<B: Backend> Default for SGD<B>

fn default() -> Self

Returns the “default value” for a type.