Struct Lars 

pub struct Lars { /* private fields */ }

Layer-wise Adaptive Rate Scaling (LARS) optimizer.

Scales the learning rate per layer using the ratio of parameter norm to gradient norm, enabling stable training with very large batch sizes.

Implementations

impl Lars

pub fn new(base_lr: f32) -> Result<Self, OptimError>

Creates LARS with required base learning rate.

pub fn with_momentum(self, momentum: f32) -> Result<Self, OptimError>

Sets momentum factor in [0, 1).

pub fn with_weight_decay(self, weight_decay: f32) -> Result<Self, OptimError>

Sets L2 weight decay factor in [0, +inf).

pub fn with_trust_coefficient( self, trust_coefficient: f32, ) -> Result<Self, OptimError>

Sets trust coefficient for the local learning rate scaling.

pub fn clear_state(&mut self)

Drops optimizer state (for example when restarting training).

pub fn learning_rate(&self) -> f32

Returns current learning rate.

pub fn set_learning_rate(&mut self, lr: f32) -> Result<(), OptimError>

Overrides current learning rate.

pub fn step( &mut self, parameter_id: u64, weights: &mut Tensor, grad: &Tensor, ) -> Result<(), OptimError>

Applies one update to raw tensor weights.

pub fn step_graph_node( &mut self, graph: &mut Graph, node: NodeId, ) -> Result<(), OptimError>

Applies one update to a trainable graph node by its NodeId.

Trait Implementations

impl Clone for Lars

fn clone(&self) -> Lars

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Lars

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LearningRate for Lars

fn learning_rate(&self) -> f32

Returns current optimizer learning rate.

fn set_learning_rate(&mut self, lr: f32) -> Result<(), OptimError>

Sets optimizer learning rate after validation.

impl StepOptimizer for Lars

fn step( &mut self, parameter_id: u64, weights: &mut Tensor, grad: &Tensor, ) -> Result<(), OptimError>