Struct AdaDelta 

pub struct AdaDelta<T: Float> { /* private fields */ }

AdaDelta optimizer configuration

AdaDelta adapts learning rates based on a moving window of gradient updates, instead of accumulating all past gradients. This eliminates the need for a manual learning rate parameter.

Key Features

• No learning rate parameter required (uses adaptive rates)
• Uses exponentially decaying average of squared gradients
• Uses exponentially decaying average of squared parameter updates
• More robust to hyperparameter choice than AdaGrad

Type Parameters

• T: Floating-point type (f32 or f64)

Implementations

impl<T: Float> AdaDelta<T>

pub fn new(rho: T, epsilon: T) -> Result<Self>

Create a new AdaDelta optimizer

Arguments

• rho: Decay rate for moving averages (typically 0.9-0.99)
• epsilon: Small constant for numerical stability (typically 1e-6 to 1e-8)

Returns

Result containing the optimizer or validation error

Example

use optirs_core::optimizers::AdaDelta;

let optimizer = AdaDelta::<f32>::new(0.95, 1e-6).expect("AdaDelta::<f32>::new succeeds");

pub fn with_warmup_boost(self, boost: T, steps: usize) -> Result<Self>

Enable an opt-in bootstrap multiplier for the first steps updates

Plain AdaDelta starts with E[Δθ²] = 0, so the first updates are on the order of sqrt(epsilon) and progress is slow until the update accumulator warms up. Setting a boost trades strict fidelity to the paper for a faster start.

The boost scales only the update that is applied to the parameters — the value accumulated into E[Δθ²] remains the unboosted update, so the adaptive learning rate stays a faithful estimate.

Errors

Returns an error if boost is not strictly positive.

pub fn warmup_boost(&self) -> T

Returns the configured warmup boost multiplier (1 when disabled)

pub fn step<'a, P, G>(&mut self, params: P, grads: G) -> Result<Array1<T>>

where P: Into<ArrayView1<'a, T>>, G: Into<ArrayView1<'a, T>>, T: 'a,

Perform a single optimization step

Arguments

• params: Current parameter values
• grads: Gradient values

Returns

Result containing updated parameters or error

Algorithm

1. Initialize accumulators on first step
2. Update exponentially decaying average of squared gradients
3. Compute RMS of gradients and previous updates
4. Compute parameter update using adaptive learning rate
5. Update exponentially decaying average of squared updates
6. Apply parameter update

Example

use optirs_core::optimizers::AdaDelta;
use scirs2_core::ndarray_ext::array;

let mut optimizer = AdaDelta::<f32>::new(0.95, 1e-6).expect("AdaDelta::<f32>::new succeeds");
let params = array![1.0, 2.0, 3.0];
let grads = array![0.1, 0.2, 0.3];

let updated_params = optimizer.step(params.view(), grads.view()).expect("optimizer.step succeeds");

pub fn step_view( &mut self, params: ArrayView1<'_, T>, grads: ArrayView1<'_, T>, ) -> Result<Array1<T>>

Perform a single optimization step on borrowed views

This is the concrete implementation behind AdaDelta::step.

pub fn step_count(&self) -> usize

Get the number of optimization steps performed

pub fn reset(&mut self)

Reset the optimizer state

Clears accumulated gradient and update history

pub fn rms_gradients(&self) -> Option<Array1<T>>

Get the current RMS of gradients for each parameter

Returns None if no steps have been performed yet

pub fn rms_updates(&self) -> Option<Array1<T>>

Get the current RMS of parameter updates

Returns None if no steps have been performed yet

Trait Implementations

impl<T: Clone + Float> Clone for AdaDelta<T>

fn clone(&self) -> AdaDelta<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug + Float> Debug for AdaDelta<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: Float> Default for AdaDelta<T>

fn default() -> Self

Returns the “default value” for a type.

impl<'de, T> Deserialize<'de> for AdaDelta<T>

where T: Deserialize<'de> + Float,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<T> Optimizer<T, Dim<[usize; 1]>> for AdaDelta<T>

where T: Float + ScalarOperand + Debug + Send + Sync,

fn get_learning_rate(&self) -> T

AdaDelta has no learning-rate hyperparameter; the effective per-parameter rate is RMS[Δθ]/RMS[g]. This reports 1 as the nominal scale.

fn set_learning_rate(&mut self, _learning_rate: T)

AdaDelta derives its step size from its accumulators, so setting a learning rate has no effect. The method exists to satisfy the Optimizer trait.

fn step( &mut self, params: &Array1<T>, gradients: &Array1<T>, ) -> Result<Array1<T>>

Updates parameters using the given gradients

fn step_list( &mut self, params_list: &[&Array<A, D>], gradients_list: &[&Array<A, D>], ) -> Result<Vec<Array<A, D>>>

Updates multiple parameter arrays at once

impl<T> Serialize for AdaDelta<T>

where T: Serialize + Float,

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.