Struct AdamW 

pub struct AdamW { /* private fields */ }

AdamW optimizer.

Attributes

• learning_rate - The learning rate of the optimizer.
• beta1 - The beta1 of the optimizer.
• beta2 - The beta2 of the optimizer.
• epsilon - The epsilon of the optimizer.
• decay - The decay of the optimizer.
• velocity - The velocity of the optimizer.
• momentum - The momentum of the optimizer.

Implementations

impl AdamW

pub fn create( learning_rate: f32, beta1: f32, beta2: f32, epsilon: f32, decay: f32, ) -> Optimizer

Creates a new AdamW optimizer.

Default values

• learning_rate - 0.001
• beta1 - 0.9
• beta2 - 0.999
• epsilon - 1e-8
• decay - 0.01

pub fn update( &mut self, layer: usize, filter: usize, bias: bool, stepnr: i32, values: &mut Tensor, gradients: &mut Tensor, )

Updates the weights of the layer. Source.

Function

• weights (values) -= learning_rate * decay * values
• momentum = beta1 * momentum + (1 - beta1) * gradients
• velocity = beta2 * velocity + (1 - beta2) * gradients^2
• m = momentum / (1 - beta1^stepnr)
• v = velocity / (1 - beta2^stepnr)
• weights (values) -= learning_rate * m / (v.sqrt() + epsilon)

Arguments

• layer - The layer of the network.
• filter - The filter of the layer. Always 0 if the layer is not a convolutional layer.
• bias - The bias of the layer.
• stepnr - The step number of the optimizer.
• values - The weights of the layer.
• gradients - The gradients of the layer.

Trait Implementations

impl Clone for AdamW

fn clone(&self) -> AdamW

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.