Struct AdamPOptimizer 

pub struct AdamPOptimizer { /* private fields */ }

AdamP optimizer with projection-based weight decay.

AdamP addresses the issue that Adam’s weight decay can slow down training on scale-invariant weights (e.g., batch normalization, layer normalization). It uses projection to apply weight decay only in the direction orthogonal to the gradient.

Key Features

• Projection-based weight decay
• Maintains Adam’s adaptive learning rates
• Better generalization on deep networks
• Particularly effective with normalization layers

Example

use tensorlogic_train::{AdamPOptimizer, OptimizerConfig};

let config = OptimizerConfig {
    learning_rate: 0.001,
    weight_decay: 0.01,  // L2 regularization
    ..Default::default()
};

let optimizer = AdamPOptimizer::new(config);

Implementations

impl AdamPOptimizer

pub fn new(config: OptimizerConfig) -> Self

Create a new AdamP optimizer with default parameters.

pub fn with_params( config: OptimizerConfig, nesterov: f64, delta: f64, wd_ratio: f64, ) -> Self

Create AdamP with custom hyperparameters.

Arguments

• config - Base optimizer configuration
• nesterov - Nesterov momentum coefficient (default: 0.9)
• delta - Projection threshold (default: 0.1)
• wd_ratio - Weight decay ratio (default: 1.0)

impl AdamPOptimizer

pub fn reset(&mut self)

Reset optimizer state (clear momentum and timestep).

Trait Implementations

impl Clone for AdamPOptimizer

fn clone(&self) -> AdamPOptimizer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Optimizer for AdamPOptimizer

fn step( &mut self, parameters: &mut HashMap<String, Array<f64, Ix2>>, gradients: &HashMap<String, Array<f64, Ix2>>, ) -> TrainResult<()>

Update parameters with computed gradients.

fn state_dict(&self) -> HashMap<String, Vec<f64>>

Get optimizer state for checkpointing.

fn load_state_dict(&mut self, state: HashMap<String, Vec<f64>>)

Load optimizer state from checkpoint.

fn get_lr(&self) -> f64

Get current learning rate.

fn set_lr(&mut self, lr: f64)

Set learning rate.

fn zero_grad(&mut self)

Zero all gradients.