Struct AdaDelta 

pub struct AdaDelta<T: Float> { /* private fields */ }

AdaDelta optimizer configuration

AdaDelta adapts learning rates based on a moving window of gradient updates, instead of accumulating all past gradients. This eliminates the need for a manual learning rate parameter.

Key Features

• No learning rate parameter required (uses adaptive rates)
• Uses exponentially decaying average of squared gradients
• Uses exponentially decaying average of squared parameter updates
• More robust to hyperparameter choice than AdaGrad

Type Parameters

• T: Floating-point type (f32 or f64)

Implementations

impl<T: Float> AdaDelta<T>

pub fn new(rho: T, epsilon: T) -> Result<Self>

Create a new AdaDelta optimizer

Arguments

• rho: Decay rate for moving averages (typically 0.9-0.99)
• epsilon: Small constant for numerical stability (typically 1e-6 to 1e-8)

Returns

Result containing the optimizer or validation error

Example

use optirs_core::optimizers::AdaDelta;

let optimizer = AdaDelta::<f32>::new(0.95, 1e-6).unwrap();

pub fn step( &mut self, params: ArrayView1<'_, T>, grads: ArrayView1<'_, T>, ) -> Result<Array1<T>>

Perform a single optimization step

Arguments

• params: Current parameter values
• grads: Gradient values

Returns

Result containing updated parameters or error

Algorithm

1. Initialize accumulators on first step
2. Update exponentially decaying average of squared gradients
3. Compute RMS of gradients and previous updates
4. Compute parameter update using adaptive learning rate
5. Update exponentially decaying average of squared updates
6. Apply parameter update

Example

use optirs_core::optimizers::AdaDelta;
use scirs2_core::ndarray_ext::array;

let mut optimizer = AdaDelta::<f32>::new(0.95, 1e-6).unwrap();
let params = array![1.0, 2.0, 3.0];
let grads = array![0.1, 0.2, 0.3];

let updated_params = optimizer.step(params.view(), grads.view()).unwrap();

pub fn step_count(&self) -> usize

Get the number of optimization steps performed

pub fn reset(&mut self)

Reset the optimizer state

Clears accumulated gradient and update history

pub fn rms_gradients(&self) -> Option<Array1<T>>

Get the current RMS of gradients for each parameter

Returns None if no steps have been performed yet

pub fn rms_updates(&self) -> Option<Array1<T>>

Get the current RMS of parameter updates

Returns None if no steps have been performed yet

Trait Implementations

impl<T: Clone + Float> Clone for AdaDelta<T>

fn clone(&self) -> AdaDelta<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug + Float> Debug for AdaDelta<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: Float> Default for AdaDelta<T>

fn default() -> Self

Returns the “default value” for a type.

impl<'de, T> Deserialize<'de> for AdaDelta<T>

where T: Deserialize<'de> + Float,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<T> Serialize for AdaDelta<T>

where T: Serialize + Float,

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.