Struct LionOptimizer 

pub struct LionOptimizer { /* private fields */ }

Lion optimizer.

Update rule:

1. c_t = β1 * m_{t-1} + (1 - β1) * g_t (interpolation)
2. θ_t = θ_{t-1} - lr * (sign(c_t) + λ * θ_{t-1}) (parameter update with weight decay)
3. m_t = β2 * m_{t-1} + (1 - β2) * g_t (momentum update)

Key differences from Adam:

• Uses sign(momentum) instead of normalized gradients
• Only tracks first moment (momentum), no second moment
• Typically requires smaller learning rates than Adam
• More memory efficient

Implementations

impl LionOptimizer

pub fn new(config: LionConfig) -> TrainResult<Self>

Create a new Lion optimizer.

pub fn step( &mut self, params: &mut HashMap<String, Array1<f64>>, gradients: &HashMap<String, Array1<f64>>, ) -> TrainResult<()>

Perform a single optimization step.

pub fn get_lr(&self) -> f64

Get the current learning rate.

pub fn set_lr(&mut self, lr: f64)

Set the learning rate.

pub fn state_dict(&self) -> HashMap<String, Vec<f64>>

Get optimizer state for checkpointing.

pub fn load_state_dict( &mut self, state: &HashMap<String, Vec<f64>>, ) -> TrainResult<()>

Load optimizer state from checkpoint.

pub fn reset(&mut self)

Reset optimizer state.