Expand description
Reverse-mode (tape / Wengert-list) automatic differentiation.
Where forward mode (see the tpt-math-autodiff-fwd crate) propagates one
derivative direction per evaluation, reverse mode records the computation
on a GradientTape and then replays it backwards, producing all
partial derivatives of a single scalar output in one pass. That makes it
the right tool for gradients of many-input, one-output functions such as
loss functions and objective functions.
§How it works
GradientTapeowns a Wengert list: an append-only sequence of nodes, each holding one primitiveOpand the primal value it produced.Variableis aCopyhandle to a node — its index, its value, and a back-pointer to the tape. Arithmetic operators and math methods evaluate the primal value, push a node, and return a handle to it.Op::diffsupplies the local derivative rule for each primitive as a small list of(operand, ∂out/∂operand)Partials.GradientTape::backwardseeds the output node with an adjoint of1and scans the tape in reverse. Because operands are always recorded before the nodes that consume them, reverse index order is a reverse topological order, so each node’s adjoint is complete by the time it is visited. Contributions are accumulated, so re-using a subexpression (a diamond in the DAG) is handled automatically.
§Examples
use tpt_math_autodiff_rev::GradientTape;
let tape = GradientTape::new();
let x = tape.var(2.0);
let y = tape.var(3.0);
let z = x * x + y.sin();
let g = tape.gradient(z, &[x, y]);
assert!((g[0] - 4.0).abs() < 1e-9); // dz/dx = 2x
assert!((g[1] - 3.0_f64.cos()).abs() < 1e-9); // dz/dy = cos(y)Supported primitives: +, -, *, / (between variables and with plain
f64 on either side), unary -, sin, cos, tan, exp,
ln, sqrt, powi, powf and recip, plus constants that
never carry a gradient.
Modules§
- prelude
- Everything needed to build and differentiate an expression.
Structs§
- Gradient
- The adjoints produced by a backward pass:
∂output/∂nodefor every node on the tape. - Gradient
Tape - A tape that records every operation performed on its
Variables so that all partial derivatives can be recovered with a single backward pass. - Partials
- A local Jacobian row: up to
MAX_OPERANDS(operand index, derivative)pairs. - Variable
- A scalar bound to a
GradientTape.
Enums§
- Op
- A single primitive operation in the tape’s Wengert list.
Constants§
- MAX_
OPERANDS - The maximum number of operands any
Opcan reference.
Functions§
- value_
and_ gradient - Evaluate
fatinputsand return(value, gradient).