Struct SGD 

pub struct SGD<A: Float + ScalarOperand + Debug> { /* private fields */ }

Stochastic Gradient Descent optimizer

Implements the classic SGD algorithm with support for momentum and weight decay.

Formula: v_t = momentum * v_{t-1} + learning_rate * (gradient + weight_decay * param) param_t = param_{t-1} - v_t

Examples

use scirs2_core::ndarray::Array1;
use optirs_core::optimizers::{SGD, Optimizer};

// Initialize parameters and gradients
let params = Array1::zeros(5);
let gradients = Array1::from_vec(vec![0.1, 0.2, -0.3, 0.0, 0.5]);

// Create an SGD optimizer with learning rate 0.01 and momentum 0.9
let mut optimizer = SGD::new_with_config(0.01, 0.9, 0.0);

// Update parameters
let new_params = optimizer.step(&params, &gradients).expect("optimizer.step succeeds");

Implementations

impl<A: Float + ScalarOperand + Debug + Send + Sync> SGD<A>

pub fn new(learning_rate: A) -> Self

Creates a new SGD optimizer with the given learning rate and no momentum/weight decay

Arguments

• learning_rate - The learning rate for parameter updates

pub fn new_with_config(learning_rate: A, momentum: A, weight_decay: A) -> Self

Creates a new SGD optimizer with the full configuration

Arguments

• learning_rate - The learning rate for parameter updates
• momentum - The momentum factor (0.0 means no momentum)
• weight_decay - The weight decay factor (L2 regularization)

pub fn set_momentum(&mut self, momentum: A) -> &mut Self

Sets the momentum factor

Arguments

• momentum - The momentum factor (0.0 means no momentum)

pub fn with_momentum(self, momentum: A) -> Self

Builder method to set momentum and return self

Arguments

• momentum - The momentum factor (0.0 means no momentum)

pub fn get_momentum(&self) -> A

Gets the current momentum factor

pub fn learning_rate(&self) -> A

Gets the current learning rate

pub fn set_weight_decay(&mut self, weight_decay: A) -> &mut Self

Sets the weight decay factor

Arguments

• weight_decay - The weight decay factor (L2 regularization)

pub fn with_weight_decay(self, weight_decay: A) -> Self

Builder method to set weight decay and return self

Arguments

• weight_decay - The weight decay factor (L2 regularization)

pub fn get_weight_decay(&self) -> A

Gets the current weight decay factor

pub fn reset(&mut self)

Drops the momentum state

pub fn step_inplace_indexed<D: Dimension>( &mut self, index: usize, params: &mut Array<A, D>, gradients: &Array<A, D>, ) -> Result<()>

Applies an SGD update in place for the parameter tensor at index

This is the allocation-free hot path: velocity and parameters are updated in a single fused Zip traversal, so no temporary arrays are created per step.

pub fn step_inplace<D: Dimension>( &mut self, params: &mut Array<A, D>, gradients: &Array<A, D>, ) -> Result<()>

Applies an SGD update in place using the state slot of the first parameter tensor

pub fn step_indexed<D: Dimension>( &mut self, index: usize, params: &Array<A, D>, gradients: &Array<A, D>, ) -> Result<Array<A, D>>

Performs an SGD update for the parameter tensor at index

Each index owns an independent momentum slot, so several parameter tensors can be optimized by a single SGD instance without their velocities mixing.

Trait Implementations

impl<A: Clone + Float + ScalarOperand + Debug> Clone for SGD<A>

fn clone(&self) -> SGD<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float + ScalarOperand + Debug> Debug for SGD<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<A, D> Optimizer<A, D> for SGD<A>

where A: Float + ScalarOperand + Debug + Send + Sync, D: Dimension,

fn step( &mut self, params: &Array<A, D>, gradients: &Array<A, D>, ) -> Result<Array<A, D>>

Updates parameters using the given gradients

fn step_list( &mut self, params_list: &[&Array<A, D>], gradients_list: &[&Array<A, D>], ) -> Result<Vec<Array<A, D>>>

Updates multiple parameter arrays at once

fn get_learning_rate(&self) -> A

Gets the current learning rate

fn set_learning_rate(&mut self, learning_rate: A)

Sets a new learning rate