Trait Optimizer 

pub trait Optimizer: Send {
    // Required methods
    fn step(
        &mut self,
        weights: &mut Array1<f32>,
        bias: &mut f32,
        weight_grad: &Array1<f32>,
        bias_grad: f32,
    );
    fn learning_rate(&self) -> f32;
    fn set_learning_rate(&mut self, lr: f32);
}

Trait for parameter optimizers.

Implementors update weights and bias given their corresponding gradients and should maintain their own internal state (moments, velocities, etc.).

Required Methods

fn step( &mut self, weights: &mut Array1<f32>, bias: &mut f32, weight_grad: &Array1<f32>, bias_grad: f32, )

Apply a gradient step.

• weight_grad: gradient w.r.t. the weight vector (same length as weights).
• bias_grad: scalar gradient w.r.t. the bias.

fn learning_rate(&self) -> f32

Return the current learning rate.

fn set_learning_rate(&mut self, lr: f32)

Override the current learning rate (called by the LR scheduler).

Implementors

impl Optimizer for AdamOptimizer

impl Optimizer for SgdOptimizer