Struct Adagrad 

pub struct Adagrad<A: Float + ScalarOperand + Debug> { /* private fields */ }

Adagrad optimizer

Implements the Adagrad optimization algorithm from the paper: “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization” by Duchi et al. (2011)

Adagrad adapts the learning rate to the parameters, performing larger updates for infrequently updated parameters and smaller updates for frequently updated parameters.

Formula: G_t = G_{t-1} + g_t^2 param_t = param_{t-1} - learning_rate * g_t / (sqrt(G_t) + epsilon)

Examples

use scirs2_core::ndarray::Array1;
use optirs_core::optimizers::{Adagrad, Optimizer};

// Initialize parameters and gradients
let params = Array1::zeros(5);
let gradients = Array1::from_vec(vec![0.1, 0.2, -0.3, 0.0, 0.5]);

// Create an Adagrad optimizer with learning rate 0.01
let mut optimizer = Adagrad::new(0.01);

// Update parameters
let new_params = optimizer.step(&params, &gradients).expect("optimizer.step succeeds");

Implementations

impl<A: Float + ScalarOperand + Debug + Send + Sync> Adagrad<A>

pub fn new(learning_rate: A) -> Self

Creates a new Adagrad optimizer with the given learning rate and default settings

Arguments

• learning_rate - The learning rate for parameter updates

pub fn new_with_config(learning_rate: A, epsilon: A, weight_decay: A) -> Self

Creates a new Adagrad optimizer with the full configuration

Arguments

• learning_rate - The learning rate for parameter updates
• epsilon - Small constant for numerical stability (default: 1e-10)
• weight_decay - Weight decay factor for L2 regularization (default: 0.0)

pub fn set_epsilon(&mut self, epsilon: A) -> &mut Self

Sets the epsilon parameter

pub fn get_epsilon(&self) -> A

Gets the epsilon parameter

pub fn set_weight_decay(&mut self, weight_decay: A) -> &mut Self

Sets the weight decay parameter

pub fn get_weight_decay(&self) -> A

Gets the weight decay parameter

pub fn reset(&mut self)

Resets the internal state of the optimizer

pub fn step_indexed<D: Dimension>( &mut self, index: usize, params: &Array<A, D>, gradients: &Array<A, D>, ) -> Result<Array<A, D>>

Performs an Adagrad update for the parameter tensor at index

Each index owns an independent accumulator, so several parameter tensors can be optimized by a single Adagrad instance without their histories mixing.

Trait Implementations

impl<A: Clone + Float + ScalarOperand + Debug> Clone for Adagrad<A>

fn clone(&self) -> Adagrad<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float + ScalarOperand + Debug> Debug for Adagrad<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<A, D> Optimizer<A, D> for Adagrad<A>

where A: Float + ScalarOperand + Debug + Send + Sync, D: Dimension,

fn step( &mut self, params: &Array<A, D>, gradients: &Array<A, D>, ) -> Result<Array<A, D>>

Updates parameters using the given gradients

fn step_list( &mut self, params_list: &[&Array<A, D>], gradients_list: &[&Array<A, D>], ) -> Result<Vec<Array<A, D>>>

Updates multiple parameter arrays at once

fn get_learning_rate(&self) -> A

Gets the current learning rate

fn set_learning_rate(&mut self, learning_rate: A)

Sets a new learning rate