Struct LARS 

pub struct LARS<A: Float> { /* private fields */ }

Layer-wise Adaptive Rate Scaling (LARS) optimizer

LARS is an optimization algorithm specifically designed for large batch training, which allows scaling up the batch size significantly without loss of accuracy. It works by adapting the learning rate per layer based on the ratio of weight norm to gradient norm.

Parameters

• learning_rate - Base learning rate
• momentum - Momentum factor (default: 0.9)
• weight_decay - Weight decay factor (default: 0.0001)
• trust_coefficient - Trust coefficient for scaling (default: 0.001)
• eps - Small constant for numerical stability (default: 1e-8)
• exclude_bias_and_norm - Whether to exclude bias and normalization layers from LARS adaptation (default: true)

Example

use scirs2_core::ndarray::Array1;
use optirs_core::optimizers::{LARS, Optimizer};

let mut optimizer = LARS::new(0.01)
    .with_momentum(0.9)
    .with_weight_decay(0.0001)
    .with_trust_coefficient(0.001);

let params = Array1::zeros(10);
let gradients = Array1::ones(10);

let updated_params = optimizer.step(&params, &gradients).unwrap();
// Parameters are automatically updated

Implementations

impl<A: Float + ScalarOperand + Debug + Send + Sync> LARS<A>

pub fn new(learning_rate: A) -> Self

Create a new LARS optimizer with the given learning rate

pub fn with_momentum(self, momentum: A) -> Self

Set the momentum factor

pub fn with_weight_decay(self, weight_decay: A) -> Self

Set the weight decay factor

pub fn with_trust_coefficient(self, trust_coefficient: A) -> Self

Set the trust coefficient

pub fn with_eps(self, eps: A) -> Self

Set the epsilon value for numerical stability

pub fn with_exclude_bias_and_norm(self, exclude_bias_and_norm: bool) -> Self

Set whether to exclude bias and normalization layers from LARS adaptation

pub fn reset(&mut self)

Reset the optimizer state

Trait Implementations

impl<A: Clone + Float> Clone for LARS<A>

fn clone(&self) -> LARS<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float> Debug for LARS<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<A: Float + ScalarOperand + Debug + Send + Sync, D: Dimension + Send + Sync> Optimizer<A, D> for LARS<A>

fn step( &mut self, params: &Array<A, D>, gradients: &Array<A, D>, ) -> Result<Array<A, D>>

Updates parameters using the given gradients

fn set_learning_rate(&mut self, learning_rate: A)

Sets a new learning rate

fn get_learning_rate(&self) -> A

Gets the current learning rate

fn step_list( &mut self, params_list: &[&Array<A, D>], gradients_list: &[&Array<A, D>], ) -> Result<Vec<Array<A, D>>>

Updates multiple parameter arrays at once