Struct SophiaOptimizer 

pub struct SophiaOptimizer { /* private fields */ }

Sophia optimizer - Second-order optimizer with Hessian diagonal estimation

Maintains three state tensors per parameter:

• m: First moment estimate (exponential moving average of gradients)
• h: Hessian diagonal estimate (EMA of element-wise gradient^2 or Hutchinson estimate)
• t: Step counter for bias correction

Implementations

impl SophiaOptimizer

pub fn new(config: OptimizerConfig) -> Self

Create a new Sophia optimizer with default Sophia configuration

pub fn with_sophia_config(config: SophiaConfig) -> Self

Create a new Sophia optimizer with custom Sophia configuration

Trait Implementations

impl Optimizer for SophiaOptimizer

fn step( &mut self, parameters: &mut HashMap<String, Array<f64, Ix2>>, gradients: &HashMap<String, Array<f64, Ix2>>, ) -> TrainResult<()>

Update parameters with computed gradients.

fn zero_grad(&mut self)

Zero all gradients.

fn get_lr(&self) -> f64

Get current learning rate.

fn set_lr(&mut self, lr: f64)

Set learning rate.

fn state_dict(&self) -> HashMap<String, Vec<f64>>

Get optimizer state for checkpointing.

fn load_state_dict(&mut self, state: HashMap<String, Vec<f64>>)

Load optimizer state from checkpoint.