Struct ASGD 

pub struct ASGD { /* private fields */ }

Averaged Stochastic Gradient Descent (ASGD) optimizer

This implements the Averaged SGD algorithm which maintains a running average of parameters during training. It can achieve better convergence properties than standard SGD in certain scenarios.

Implementations

impl ASGD

pub fn new( params: Vec<Arc<RwLock<Tensor>>>, lr: Option<f32>, alpha: Option<f32>, t0: Option<f32>, lambd: Option<f32>, weight_decay: Option<f32>, ) -> Self

Create a new ASGD optimizer

Arguments

• params - Parameters to optimize
• lr - Learning rate (default: 1e-2)
• alpha - Power for computing average (default: 0.75)
• t0 - Point at which to start averaging (default: 1e6)
• lambd - Decay term (default: 1e-4)
• weight_decay - Weight decay (L2 penalty) (default: 0.0)

Trait Implementations

impl Optimizer for ASGD

fn step(&mut self) -> OptimizerResult<()>

Perform a single optimization step

fn zero_grad(&mut self)

Zero all gradients

fn get_lr(&self) -> Vec<f32>

Get the current learning rate

fn set_lr(&mut self, lr: f32)

Set one learning rate for every parameter group (broadcast).

fn add_param_group( &mut self, params: Vec<Arc<RwLock<Tensor>>>, options: HashMap<String, f32>, )

Add a parameter group

fn parameters(&self) -> Vec<Arc<RwLock<Tensor>>>

Get the parameter tensors managed by this optimizer.

fn state_dict(&self) -> OptimizerResult<OptimizerState>

Get state dict for serialization

fn load_state_dict(&mut self, state: OptimizerState) -> OptimizerResult<()>

Load state dict

fn set_lrs(&mut self, lrs: &[f32])

Set the learning rate of each parameter group individually.