Enum LazyAdamMode 

pub enum LazyAdamMode {
    Lazy,
    DormancyDecay,
}

Semantics used by SparseAdam for catching up the moment estimates of a coordinate that has been dormant. See the module-level documentation for the exact update equations of each variant.

Variants

Lazy

Pure TensorFlow LazyAdam: moments decay by a single factor of beta regardless of dormancy. Cheapest; does not model the EMA decay that elapsed while the coordinate was untouched.

DormancyDecay

Lazy Adam with dormancy catch-up: moments are decayed by beta^(t - s) (where s is the last touched step) before the current gradient is incorporated, exactly reproducing the moments that dense Adam — fed zero gradients on the dormant steps — would hold.

Trait Implementations

impl Clone for LazyAdamMode

fn clone(&self) -> LazyAdamMode

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for LazyAdamMode

impl Debug for LazyAdamMode

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for LazyAdamMode

impl PartialEq for LazyAdamMode

fn eq(&self, other: &LazyAdamMode) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for LazyAdamMode