Struct AdanConfig 

pub struct AdanConfig { /* private fields */ }

Adan Configuration.

See:

• Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models.

Implementations

impl AdanConfig

pub fn new() -> Self

Create a new instance of the config.

Arguments

Optional Arguments

grad_clipping

Gradient Clipping config.

• Defaults to None

Default Arguments

beta_1

Parameter for the first moment.

• Defaults to 0.98

beta_2

Parameter for the gradient-difference momentum.

• Defaults to 0.92

beta_3

Parameter for the second moment.

• Defaults to 0.99

epsilon

A value required for numerical stability.

• Defaults to 1e-8

weight_decay

Weight decay factor.

• Defaults to 0.0

no_prox

Disable proximal weight decay and use the decoupled update instead.

• Defaults to false

impl AdanConfig

pub fn with_beta_1(self, beta_1: f32) -> Self

Sets the value for the field beta_1.

Parameter for the first moment.

• Defaults to 0.98

pub fn with_beta_2(self, beta_2: f32) -> Self

Sets the value for the field beta_2.

Parameter for the gradient-difference momentum.

• Defaults to 0.92

pub fn with_beta_3(self, beta_3: f32) -> Self

Sets the value for the field beta_3.

Parameter for the second moment.

• Defaults to 0.99

pub fn with_epsilon(self, epsilon: f32) -> Self

Sets the value for the field epsilon.

A value required for numerical stability.

• Defaults to 1e-8

pub fn with_weight_decay(self, weight_decay: f32) -> Self

Sets the value for the field weight_decay.

Weight decay factor.

• Defaults to 0.0

pub fn with_no_prox(self, no_prox: bool) -> Self

Sets the value for the field no_prox.

Disable proximal weight decay and use the decoupled update instead.

• Defaults to false

pub fn with_grad_clipping( self, grad_clipping: Option<GradientClippingConfig>, ) -> Self

Sets the value for the field grad_clipping.

Gradient Clipping config.

• Defaults to None

impl AdanConfig

pub fn build(&self) -> Adan

Build an Adan from the config.

pub fn init<B: AutodiffBackend, M: AutodiffModule<B>>( &self, ) -> OptimizerAdaptor<Adan, M, B>

Initialize Adan optimizer.

Returns

Returns an optimizer that can be used to optimize a module.

Trait Implementations

impl Clone for AdanConfig

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Config for AdanConfig

fn save<P>(&self, file: P) -> Result<(), Error>

where P: AsRef<Path>,

Available on crate feature std only.

Saves the configuration to a file.

fn load<P>(file: P) -> Result<Self, ConfigError>

where P: AsRef<Path>,

Available on crate feature std only.

Loads the configuration from a file.

fn load_binary(data: &[u8]) -> Result<Self, ConfigError>

Loads the configuration from a binary buffer.

impl Debug for AdanConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for AdanConfig

fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for AdanConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Serialize for AdanConfig

fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.