yscv-optim 0.1.8

Optimizers (SGD, Adam, LAMB), LR schedulers, and gradient clipping
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

# yscv-optim

Optimizers, learning rate schedulers, and gradient clipping for neural network training.

```rust,ignore
use yscv_optim::*;

let mut optimizer = Adam::new(parameters, 1e-3);
let scheduler = CosineScheduler::new(100, 1e-3, 1e-6);

for epoch in 0..100 {
    optimizer.set_lr(scheduler.get_lr(epoch));
    optimizer.step();
    optimizer.zero_grad();
}
```

## Optimizers (8 + Lookahead meta-optimizer)

`Sgd`, `Adam`, `AdamW`, `RmsProp`, `RAdam`, `Lars`, `Lamb`, `Adagrad`, plus `Lookahead<O>` which wraps any of them.

## LR Schedulers (11)

`StepLr`, `MultiStepLr`, `ExponentialLr`, `CosineAnnealingLr`, `CosineAnnealingWarmRestarts`, `LinearWarmupLr`, `PolynomialDecayLr`, `OneCycleLr`, `ReduceLrOnPlateau`, `CyclicLr`, `LambdaLr`.

## Gradient Clipping

- `clip_grad_norm` — L2 norm clipping
- `clip_grad_value` — element-wise value clipping

## Tests

76 tests covering optimizer convergence, scheduler curves, clipping behavior.