Struct LookaheadOptimizer 

pub struct LookaheadOptimizer<O: Optimizer> { /* private fields */ }

Lookahead optimizer (wrapper that uses slow and fast weights).

Maintains two sets of weights: fast weights updated by an inner optimizer, and slow weights that are periodically updated as an exponential moving average.

Reference: Zhang et al., “Lookahead Optimizer: k steps forward, 1 step back”, NeurIPS 2019

Implementations

impl<O: Optimizer> LookaheadOptimizer<O>

pub fn new(inner_optimizer: O, alpha: f64, k: usize) -> TrainResult<Self>

Create a new Lookahead optimizer.

Arguments

• inner_optimizer - The inner optimizer (e.g., Adam, SGD)
• alpha - Interpolation coefficient for slow weight update (typically 0.5)
• k - Number of fast updates before slow weight synchronization (typically 5-10)

Trait Implementations

impl<O: Debug + Optimizer> Debug for LookaheadOptimizer<O>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<O: Optimizer> Optimizer for LookaheadOptimizer<O>

fn step( &mut self, parameters: &mut HashMap<String, Array<f64, Ix2>>, gradients: &HashMap<String, Array<f64, Ix2>>, ) -> TrainResult<()>

Update parameters with computed gradients.

fn zero_grad(&mut self)

Zero all gradients.

fn get_lr(&self) -> f64

Get current learning rate.

fn set_lr(&mut self, lr: f64)

Set learning rate.

fn state_dict(&self) -> HashMap<String, Vec<f64>>

Get optimizer state for checkpointing.

fn load_state_dict(&mut self, state: HashMap<String, Vec<f64>>)

Load optimizer state from checkpoint.