Struct SGD 

pub struct SGD {
    pub learning_rate: f32,
    pub momentum: f32,
    pub weight_decay: f32,
    pub dampening: f32,
    pub nesterov: bool,
    pub maximize: bool,
    pub bias: Vec<Tensor>,
}

Stochastic gradient descent optimizer

Fields

learning_rate: f32

learning rate (default: 0.001)

momentum: f32

momentum factor (default: 0.0)

weight_decay: f32

weight decay (L2 penalty) (default: 0.0)

dampening: f32

dampening for momentum (default: 0.0)

nesterov: bool

enables Nesterov momentum (default: false)

maximize: bool

maximize the objective with respect to the params, instead of minimizing (default: false)

bias: Vec<Tensor>

stores momentum, starts empty and will be initialized on demand

Implementations

impl SGD

pub fn update<'a>( &mut self, parameters: impl IntoIterator<Item = &'a mut Tensor>, gradients: impl IntoIterator<Item = Option<Tensor>>, )

Updates parameters with gradients. Number of parameters must be the same as number of gradients. Gradients can be None, those are simply skipped.

Trait Implementations

impl Default for SGD

fn default() -> Self

Returns the “default value” for a type.

impl<'a> IntoIterator for &'a SGD

type Item = &'a Tensor

The type of the elements being iterated over.

type IntoIter = IntoIter<&'a Tensor>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl<'a> IntoIterator for &'a mut SGD

type Item = &'a mut Tensor

The type of the elements being iterated over.

type IntoIter = IntoIter<&'a mut Tensor>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl Module for SGD

fn iter<'a>(&'a self) -> impl Iterator<Item = &'a Tensor>

Iterate over all tensors immutably

fn iter_mut<'a>(&'a mut self) -> impl Iterator<Item = &'a mut Tensor>

Iterate over all tensors mutably

fn iter_tensors<'a>(&'a self) -> impl Iterator<Item = (String, &'a Tensor)>

Iterate over tensors without consuming the module

fn iter_tensors_mut<'a>( &'a mut self, ) -> impl Iterator<Item = (String, &'a mut Tensor)>

From tensors

fn realize(&self) -> Result<(), ZyxError>

Realize all tensors in the module

fn set_params(&mut self, params: &mut HashMap<String, Tensor>)

Set parameters, removes them from params, skips parameters that are not found in params.

fn save(&self, path: impl AsRef<Path>) -> Result<(), ZyxError>

Save tensors or modules to a file determined by file extension. Currently only safetensors is supported format.