Struct SGD 

pub struct SGD { /* private fields */ }

Stochastic Gradient Descent optimizer.

SGD updates parameters using the gradient of the loss function:

θ = θ - η * ∇L(θ)

With momentum:

v = γ * v + η * ∇L(θ)
θ = θ - v

where:

• θ is the parameter vector
• η is the learning rate
• γ is the momentum coefficient
• v is the velocity vector
• ∇L(θ) is the gradient of the loss

Example

use aprender::optim::SGD;
use aprender::primitives::Vector;

// Create SGD with momentum
let mut optimizer = SGD::new(0.1).with_momentum(0.9);

let mut params = Vector::from_slice(&[0.0, 0.0]);
let gradients = Vector::from_slice(&[1.0, 2.0]);

// First step
optimizer.step(&mut params, &gradients);

// With momentum, velocity builds up over iterations
optimizer.step(&mut params, &gradients);

Implementations

impl SGD

pub fn new(learning_rate: f32) -> Self

Creates a new SGD optimizer with the given learning rate.

Arguments

• learning_rate - Step size for parameter updates

Example

use aprender::optim::SGD;

let optimizer = SGD::new(0.01);
assert!((optimizer.learning_rate() - 0.01).abs() < 1e-6);

pub fn with_momentum(self, momentum: f32) -> Self

Sets the momentum coefficient.

Momentum helps accelerate SGD in the relevant direction and dampens oscillations. Typical values are 0.9 or 0.99.

Arguments

• momentum - Momentum coefficient between 0.0 and 1.0

Example

use aprender::optim::SGD;

let optimizer = SGD::new(0.01).with_momentum(0.9);
assert!((optimizer.momentum() - 0.9).abs() < 1e-6);

pub fn learning_rate(&self) -> f32

Returns the learning rate.

pub fn momentum(&self) -> f32

Returns the momentum coefficient.

pub fn step(&mut self, params: &mut Vector<f32>, gradients: &Vector<f32>)

Updates parameters using gradients.

If momentum is enabled, maintains velocity vectors for each parameter.

Arguments

• params - Mutable reference to parameter vector
• gradients - Gradient vector (same length as params)

Panics

Panics if params and gradients have different lengths.

Example

use aprender::optim::SGD;
use aprender::primitives::Vector;

let mut optimizer = SGD::new(0.1);
let mut params = Vector::from_slice(&[1.0, 2.0]);
let gradients = Vector::from_slice(&[0.5, 1.0]);

optimizer.step(&mut params, &gradients);

// params = [1.0 - 0.1*0.5, 2.0 - 0.1*1.0] = [0.95, 1.9]
assert!((params[0] - 0.95).abs() < 1e-6);
assert!((params[1] - 1.9).abs() < 1e-6);

pub fn reset(&mut self)

Resets the optimizer state (velocity vectors).

Call this when starting training on a new model or after significant changes to the optimization problem.

Example

use aprender::optim::SGD;
use aprender::primitives::Vector;

let mut optimizer = SGD::new(0.1).with_momentum(0.9);
let mut params = Vector::from_slice(&[1.0]);
let gradients = Vector::from_slice(&[1.0]);

optimizer.step(&mut params, &gradients);
optimizer.reset();

// Velocity is now reset to zero

pub fn has_momentum(&self) -> bool

Returns whether momentum is enabled.

Trait Implementations

impl Clone for SGD

fn clone(&self) -> SGD

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SGD

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for SGD

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Optimizer for SGD

fn step(&mut self, params: &mut Vector<f32>, gradients: &Vector<f32>)

Stochastic update (mini-batch mode) - for SGD, Adam, RMSprop.

fn reset(&mut self)

Resets the optimizer state (momentum, history, etc.).

fn minimize<F, G>( &mut self, _objective: F, _gradient: G, _x0: Vector<f32>, ) -> OptimizationResult

where F: Fn(&Vector<f32>) -> f32, G: Fn(&Vector<f32>) -> Vector<f32>,

Batch optimization (deterministic mode) - for L-BFGS, CG, Damped Newton.

impl Serialize for SGD

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.