1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
// SPDX-License-Identifier: MIT OR Apache-2.0
//! # fuel-training
//!
//! **Layer**: Training | **Stability**: experimental
//!
//! Training orchestration for the Fuel ML framework. This crate is the
//! canonical home for training-loop infrastructure on top of the lazy
//! substrate in `fuel-core` (Optimizer, Var, VarMap, the
//! `lazy_training_augmentations` schedulers and grad clippers).
//!
//! ## Modules
//!
//! - [`lr_scheduler`] — Learning rate schedulers (cosine annealing, warmup,
//! step decay, sequential composition).
//! - [`grad_clip`] — Gradient clipping (global L2 norm, per-element value).
//! - [`grad_accum`] — Gradient accumulation for simulating larger batch sizes.
//! - [`checkpoint`] — Checkpoint save/load with training metadata (epoch, step,
//! metrics) for resumable training.
//! - [`training_loop`] — Composable training loop driver that wires together
//! clipping, scheduling, and logging.
//! - [`train`] — `Parameter`/`OptimizerConfig`/`TrainState`: the lazy-DAG
//! parameter-storage and SGD/AdamW step driver (moved from
//! `fuel-core::train`, board #109 follow-up — `fuel::train` is
//! retired, this is its new home). Ships its own `LrSchedule` trait
//! (`ConstLr`/`WarmupCosine`/`WarmupLinear`) distinct from
//! [`training_loop`]'s `fuel::lazy_training_augmentations` schedulers —
//! a known, tracked overlap (see `docs/gaps.md`), not reconciled here.
//!
//! ## What is NOT here
//!
//! - Model definitions (stay in `fuel-transformers`)
//! - Inference-specific code (use `fuel-inference`)
//! - Dataset loading or preprocessing (use `fuel-datasets`)
//! - Optimiser kernels (stay in `fuel-nn`)
//!
//! ## Layer placement
//!
//! ```text
//! fuel-training ← you are here (training orchestration)
//! fuel-nn (layers, optimisers, VarBuilder)
//! fuel-core (tensors, devices, autograd)
//! ```
//!
//! Nothing in `fuel-core` depends on this crate. It is a leaf that
//! aggregates; it does not define.
//!
//! ## Example
//!
//! ```rust,no_run
//! use fuel_nn::optim::{AdamW, Optimizer, Var, AdamWConfig};
//! use fuel::lazy_training_augmentations::CosineSchedule;
//! use fuel_training::training_loop::TrainingLoop;
//!
//! # fn main() -> fuel::Result<()> {
//! let x = Var::new("x", fuel::Shape::from_dims(&[3]), vec![1.0_f32, 2.0, 3.0])?;
//! let mut opt = AdamW::new(vec![x.clone()], AdamWConfig::default())?;
//! let sched = CosineSchedule { base_lr: 1e-3, warmup_steps: 10, total_steps: 100 };
//! let mut loop_ = TrainingLoop::new()
//! .with_max_grad_norm(1.0)
//! .with_scheduler(sched);
//! // Per step: build a `Tensor` loss using `x.tensor(&anchor)`, then
//! // `loop_.step(&loss, &mut opt)?;`
//! # Ok(())
//! # }
//! ```
// Re-exports from the lazy training substrate in fuel-core. These used to
// live in fuel-training as eager wrappers around fuel-nn's Optimizer; the
// lazy equivalents have strictly broader coverage so the eager modules are
// retired and only re-exported here for source-compatibility.