Struct Lookahead 

pub struct Lookahead<A, O, D>
where
    A: Float + ScalarOperand + Debug,
    O: Optimizer<A, D> + Clone,
    D: Dimension,
{ /* private fields */ }

Lookahead optimizer

Implements the “Lookahead Optimizer: k steps forward, 1 step back” algorithm. This optimizer maintains two sets of weights: “fast” weights that are updated by an inner optimizer, and “slow” weights that follow behind at a controlled pace.

The algorithm proceeds by:

1. Starting with both sets of weights synchronized
2. Letting the fast weights explore using the inner optimizer for k steps
3. Then updating the slow weights to move partially toward the fast weights
4. Resetting the fast weights back to the slow weights
5. Repeating this process

This provides more stable optimization by allowing aggressive exploration while maintaining a conservative trajectory.

Parameters

• inner_optimizer - The optimizer to use for fast weight updates
• alpha - The step size for slow weight updates (default: 0.5)
• k - The number of fast weight updates before updating slow weights (default: 5)

Example

use scirs2_core::ndarray::Array1;
use optirs_core::optimizers::{Lookahead, SGD};
use optirs_core::Optimizer;

// Create an inner optimizer
let sgd = SGD::new(0.01);

// Wrap it with Lookahead
let mut optimizer = Lookahead::new(sgd);

// Use like any other optimizer
let params = Array1::zeros(10);
let gradients = Array1::ones(10);
let updated_params = optimizer.step(&params, &gradients).unwrap();

Implementations

impl<A, O, D> Lookahead<A, O, D>

where A: Float + ScalarOperand + Debug, O: Optimizer<A, D> + Clone, D: Dimension,

pub fn new(inner_optimizer: O) -> Self

Creates a new Lookahead optimizer with the given inner optimizer and default settings

pub fn with_config(inner_optimizer: O, alpha: A, k: usize) -> Self

Creates a new Lookahead optimizer with the specified alpha and k values

pub fn with_alpha(self, alpha: A) -> Self

Set the alpha parameter (slow weights step size)

pub fn with_k(self, k: usize) -> Self

Set the k parameter (synchronization period)

pub fn inner_optimizer(&self) -> &O

Get the inner optimizer

pub fn inner_optimizer_mut(&mut self) -> &mut O

Get a mutable reference to the inner optimizer

pub fn alpha(&self) -> A

Get the alpha parameter (slow weights step size)

pub fn k(&self) -> usize

Get the k parameter (synchronization period)

pub fn use_slow_weights_for_eval(&mut self)

Switches to using slow weights for evaluation Call this before evaluation to get better performance

pub fn use_fast_weights_for_train(&mut self)

Switches to using fast weights for training Call this after evaluation to resume training

pub fn reset(&mut self)

Resets the internal state

Trait Implementations

impl<A, O, D> Clone for Lookahead<A, O, D>

where A: Float + ScalarOperand + Debug, O: Optimizer<A, D> + Clone, D: Dimension,

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A, O, D> Debug for Lookahead<A, O, D>

where A: Float + ScalarOperand + Debug, O: Optimizer<A, D> + Clone + Debug, D: Dimension,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<A, O, D> Optimizer<A, D> for Lookahead<A, O, D>

where A: Float + ScalarOperand + Debug + Send + Sync, O: Optimizer<A, D> + Clone + Send + Sync, D: Dimension,

fn step( &mut self, params: &Array<A, D>, gradients: &Array<A, D>, ) -> Result<Array<A, D>>

Updates parameters using the given gradients

fn set_learning_rate(&mut self, learning_rate: A)

Sets a new learning rate

fn get_learning_rate(&self) -> A

Gets the current learning rate

fn step_list( &mut self, params_list: &[&Array<A, D>], gradients_list: &[&Array<A, D>], ) -> Result<Vec<Array<A, D>>>

Updates multiple parameter arrays at once