Enum DecayStrategy 

pub enum DecayStrategy<A: Float + Debug> {
    Linear {
        final_lr: A,
    },
    Exponential {
        decay_rate: A,
    },
    Step {
        decay_rate: A,
        step_size: usize,
    },
    Cosine {
        min_lr: A,
    },
    Constant,
}

Decay strategy to use after the warmup phase

Variants

Linear

Linear decay to a final value

Fields

final_lr: A

Final learning rate

Exponential

Exponential decay with a given decay rate

Fields

decay_rate: A

Decay rate per step

Step

Step decay with a given step size and decay rate

Fields

decay_rate: A

Decay rate at each step

step_size: usize

Step size (number of steps between decays)

Cosine

Cosine annealing to a minimum value

Fields

min_lr: A

Minimum learning rate

Constant

No decay - constant learning rate after warmup

Trait Implementations

impl<A: Clone + Float + Debug> Clone for DecayStrategy<A>

fn clone(&self) -> DecayStrategy<A>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<A: Debug + Float + Debug> Debug for DecayStrategy<A>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.