Expand description
Optimizers and training helpers for yscv models.
Structs§
- Adagrad
- Adagrad optimizer with optional L2 weight decay.
- Adam
- Adam optimizer with optional L2 weight decay.
- AdamW
- AdamW optimizer with decoupled weight decay.
- Cosine
Annealing Lr - Cosine annealing learning-rate scheduler.
- Cosine
Annealing Warm Restarts - Cosine annealing with warm restarts learning-rate scheduler.
- Cyclic
Lr - Cyclic learning-rate scheduler with triangular policy.
- Exponential
Lr - Exponential learning-rate scheduler.
- Lamb
- Layer-wise Adaptive Moments optimizer for Batch training (LAMB).
- Lambda
Lr - Lambda learning-rate scheduler.
- Lars
- Layer-wise Adaptive Rate Scaling (LARS) optimizer.
- Linear
Warmup Lr - Linear warmup learning-rate scheduler.
- Lookahead
- Lookahead optimizer wrapper.
- Multi
Step Lr - Multi-step learning-rate scheduler.
- OneCycle
Lr - One-cycle learning-rate scheduler with linear warmup and linear cooldown.
- Polynomial
Decay Lr - Polynomial decay learning-rate scheduler.
- RAdam
- RAdam (Rectified Adam) optimizer with variance rectification.
- Reduce
LrOn Plateau - Reduce learning rate when a metric has stopped improving.
- RmsProp
- RMSProp optimizer with optional momentum, weight decay, and centered variance.
- Sgd
- Stochastic gradient descent optimizer with optional momentum and weight decay.
- StepLr
- Piecewise constant learning-rate scheduler.
Enums§
- Optim
Error - Errors returned by optimizer configuration and update steps.
Constants§
Traits§
- Learning
Rate - Shared learning-rate control surface for optimizers.
- LrScheduler
- Scheduler abstraction for stateful learning-rate policies.
- Step
Optimizer - Trait for optimizers that support a per-parameter
stepupdate.
Functions§
- clip_
grad_ norm_ - Clips the total norm of gradients for the given nodes in-place.
- clip_
grad_ value_ - Clamps every gradient element to the range
[-max_val, max_val]in-place.