Struct AdaBound 

pub struct AdaBound<T: Float> { /* private fields */ }

AdaBound optimizer configuration

AdaBound combines the benefits of adaptive learning rate methods (like Adam) with the strong generalization of SGD by dynamically bounding the learning rates.

Key Features

• Smooth transition from Adam to SGD during training
• Dynamic bounds prevent learning rates from becoming too large or too small
• Better generalization than pure Adam
• Maintains fast convergence of adaptive methods

Type Parameters

• T: Floating-point type (f32 or f64)

Implementations

impl<T: Float + ScalarOperand> AdaBound<T>

pub fn new( learning_rate: T, final_lr: T, beta1: T, beta2: T, epsilon: T, gamma: T, weight_decay: T, amsbound: bool, ) -> Result<Self>

Create a new AdaBound optimizer

Arguments

• learning_rate: Initial learning rate (typically 0.001)
• final_lr: Final learning rate for SGD convergence (typically 0.1)
• beta1: First moment decay rate (typically 0.9)
• beta2: Second moment decay rate (typically 0.999)
• epsilon: Small constant for numerical stability (typically 1e-8)
• gamma: Convergence speed parameter (typically 1e-3)
• weight_decay: L2 regularization coefficient (typically 0.0)
• amsbound: Use AMSBound variant if true

Example

use optirs_core::optimizers::AdaBound;

let optimizer = AdaBound::<f32>::new(
    0.001,  // learning_rate
    0.1,    // final_lr
    0.9,    // beta1
    0.999,  // beta2
    1e-8,   // epsilon
    1e-3,   // gamma
    0.0,    // weight_decay
    false   // amsbound
).unwrap();

pub fn step( &mut self, params: ArrayView1<'_, T>, grads: ArrayView1<'_, T>, ) -> Result<Array1<T>>

Perform a single optimization step

Arguments

• params: Current parameter values
• grads: Gradient values

Returns

Result containing updated parameters or error

Algorithm

1. Initialize moments on first step
2. Apply weight decay if configured
3. Update biased first moment: m_t = β₁ * m_{t-1} + (1 - β₁) * g_t
4. Update biased second moment: v_t = β₂ * v_{t-1} + (1 - β₂) * g_t²
5. Compute bias-corrected moments
6. Compute dynamic bounds: [α_l(t), α_u(t)]
7. Compute clipped learning rate per parameter
8. Apply parameter update: θ_{t+1} = θ_t - η_t * m̂_t

Example

use optirs_core::optimizers::AdaBound;
use scirs2_core::ndarray_ext::array;

let mut optimizer = AdaBound::<f32>::default();
let params = array![1.0, 2.0, 3.0];
let grads = array![0.1, 0.2, 0.3];

let updated_params = optimizer.step(params.view(), grads.view()).unwrap();

pub fn step_count(&self) -> usize

Get the number of optimization steps performed

pub fn reset(&mut self)

Reset the optimizer state

pub fn current_bounds(&self) -> (T, T)

Get current dynamic bounds [lower, upper]

Trait Implementations

impl<T: Clone + Float> Clone for AdaBound<T>

fn clone(&self) -> AdaBound<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug + Float> Debug for AdaBound<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: Float + ScalarOperand> Default for AdaBound<T>

fn default() -> Self

Returns the “default value” for a type.

impl<'de, T> Deserialize<'de> for AdaBound<T>

where T: Deserialize<'de> + Float,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<T> Serialize for AdaBound<T>

where T: Serialize + Float,

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.