Struct SimdSGD 

pub struct SimdSGD<A: Float> { /* private fields */ }

SIMD-accelerated Stochastic Gradient Descent optimizer

This is a specialized version of SGD optimized for 1D arrays using SIMD operations. For maximum performance, use this when working with flattened parameter vectors.

Formula: v_t = momentum * v_{t-1} + learning_rate * (gradient + weight_decay * param) param_t = param_{t-1} - v_t

Performance

This implementation uses SIMD instructions (AVX2/SSE/NEON) for:

• Parameter updates
• Momentum computation
• Weight decay application

Expected speedup: 2-4x over scalar implementation for large parameter arrays

Examples

use scirs2_core::ndarray::Array1;
use optirs_core::optimizers::{SimdSGD, Optimizer};

// Initialize parameters and gradients
let params = Array1::zeros(1000);
let gradients = Array1::from_elem(1000, 0.1);

// Create SIMD-accelerated SGD optimizer
let mut optimizer = SimdSGD::new(0.01);
optimizer.set_momentum(0.9);

// Update parameters with SIMD acceleration
let new_params = optimizer.step(&params, &gradients).unwrap();

Implementations

impl<A: Float> SimdSGD<A>

pub fn new(learning_rate: A) -> Self

Creates a new SIMD-accelerated SGD optimizer

Arguments

• learning_rate - The learning rate for parameter updates

pub fn new_with_config(learning_rate: A, momentum: A, weight_decay: A) -> Self

Creates a new SIMD SGD optimizer with full configuration

Arguments

• learning_rate - The learning rate for parameter updates
• momentum - The momentum factor (0.0 means no momentum)
• weight_decay - The weight decay factor (L2 regularization)

pub fn set_momentum(&mut self, momentum: A) -> &mut Self

Sets the momentum factor

pub fn with_momentum(self, momentum: A) -> Self

Builder method to set momentum and return self

pub fn get_momentum(&self) -> A

Gets the current momentum factor

pub fn learning_rate(&self) -> A

Gets the current learning rate

pub fn set_weight_decay(&mut self, weight_decay: A) -> &mut Self

Sets the weight decay factor

pub fn with_weight_decay(self, weight_decay: A) -> Self

Builder method to set weight decay and return self

pub fn get_weight_decay(&self) -> A

Gets the current weight decay factor

pub fn reset(&mut self)

Resets the optimizer state

Trait Implementations

impl<A: Clone + Float> Clone for SimdSGD<A>

fn clone(&self) -> SimdSGD<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float> Debug for SimdSGD<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer<f32, Dim<[usize; 1]>> for SimdSGD<f32>

fn step( &mut self, params: &Array1<f32>, gradients: &Array1<f32>, ) -> Result<Array1<f32>>

Updates parameters using the given gradients

fn get_learning_rate(&self) -> f32

Gets the current learning rate

fn set_learning_rate(&mut self, learning_rate: f32)

Sets a new learning rate

fn step_list( &mut self, params_list: &[&Array<A, D>], gradients_list: &[&Array<A, D>], ) -> Result<Vec<Array<A, D>>>

Updates multiple parameter arrays at once

impl Optimizer<f64, Dim<[usize; 1]>> for SimdSGD<f64>

fn step( &mut self, params: &Array1<f64>, gradients: &Array1<f64>, ) -> Result<Array1<f64>>

Updates parameters using the given gradients

fn get_learning_rate(&self) -> f64

Gets the current learning rate

fn set_learning_rate(&mut self, learning_rate: f64)

Sets a new learning rate

fn step_list( &mut self, params_list: &[&Array<A, D>], gradients_list: &[&Array<A, D>], ) -> Result<Vec<Array<A, D>>>

Updates multiple parameter arrays at once