Struct Adam 

pub struct Adam { /* private fields */ }

Adam optimizer (Adaptive Moment Estimation).

Combines momentum with adaptive learning rates per parameter.

Mathematical Description

$$m_t = \beta_1 m_{t-1} + (1-\beta_1) g_t$$ $$v_t = \beta_2 v_{t-1} + (1-\beta_2) g_t^2$$ $$\hat{m}_t = m_t / (1 - \beta_1^t)$$ $$\hat{v}t = v_t / (1 - \beta_2^t)$$ $$\theta_t = \theta{t-1} - \alpha \hat{m}_t / (\sqrt{\hat{v}_t} + \epsilon)$$

Example

use logosq_optimizer::Adam;

let optimizer = Adam::new()
    .with_learning_rate(0.001)
    .with_beta1(0.9)
    .with_beta2(0.999);

Implementations

impl Adam

pub fn new() -> Self

Create a new Adam optimizer with default parameters.

pub fn with_learning_rate(self, lr: f64) -> Self

Set the learning rate.

pub fn with_beta1(self, beta1: f64) -> Self

Set beta1 (first moment decay).

pub fn with_beta2(self, beta2: f64) -> Self

Set beta2 (second moment decay).

pub fn with_epsilon(self, epsilon: f64) -> Self

Set epsilon for numerical stability.

pub fn with_gradient_clip(self, max_norm: f64) -> Self

Enable gradient clipping.

pub fn minimize<F: ObjectiveFunction>( &self, objective: &F, initial_params: &[f64], criteria: &ConvergenceCriteria, ) -> Result<OptimizationResult, OptimizerError>

Minimize an objective function.

Trait Implementations

impl Default for Adam

fn default() -> Self

Returns the “default value” for a type.

impl Optimizer for Adam

fn step( &self, params: &mut [f64], gradients: &[f64], iteration: usize, ) -> Result<(), OptimizerError>

Perform a single optimization step.

fn name(&self) -> &str

Get the optimizer name.

fn reset(&mut self)

Reset internal state (for optimizers with momentum).

fn current_learning_rate(&self, _iteration: usize) -> f64

Get current learning rate (may be adaptive).