Struct tch::nn::Optimizer

pub struct Optimizer<T> { /* fields omitted */ }

An optimizer to run gradient descent.

Implementations

impl<T> Optimizer<T>

pub fn zero_grad(&mut self)

Zeroes the gradient for the tensors tracked by this optimizer.

pub fn clip_grad_value(&self, max: f64)

Clips gradient value at some specified maximum value.

pub fn clip_grad_norm(&self, max: f64)

Clips gradient L2 norm over all trainable parameters.

The norm is computed over all gradients together, as if they were concatenated into a single vector.

pub fn step(&mut self)

Performs an optimization step, updating the tracked tensors based on their gradients.

pub fn backward_step(&mut self, loss: &Tensor)

Applies a backward step pass, update the gradients, and performs an optimization step.

pub fn backward_step_clip(&mut self, loss: &Tensor, max: f64)

Applies a backward step pass, update the gradients, and performs an optimization step.

The gradients are clipped based on max before being applied.

pub fn backward_step_clip_norm(&mut self, loss: &Tensor, max: f64)

Applies a backward step pass, update the gradients, and performs an optimization step.

The gradients L2 norm is clipped based on max.

pub fn set_lr(&mut self, lr: f64)

Sets the optimizer learning rate.

pub fn set_momentum(&mut self, m: f64)

Sets the optimizer momentum.

pub fn set_lr_group(&mut self, group: usize, lr: f64)

Sets the optimizer learning rate for a parameter group.

pub fn set_momentum_group(&mut self, group: usize, m: f64)

Sets the optimizer momentum.

pub fn trainable_variables(&self) -> Vec<Tensor>

Returns all the trainable variables for this optimizer.

pub fn set_weight_decay(&mut self, weight_decay: f64)

Sets the optimizer weight decay.

pub fn set_weight_decay_group(&mut self, group: usize, weight_decay: f64)

Sets the optimizer weight decay.

Trait Implementations

impl<T: Debug> Debug for Optimizer<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.