Skip to main content

Crate tpt_math_autodiff_rev

Crate tpt_math_autodiff_rev 

Source
Expand description

Reverse-mode (tape / Wengert-list) automatic differentiation.

Where forward mode (see the tpt-math-autodiff-fwd crate) propagates one derivative direction per evaluation, reverse mode records the computation on a GradientTape and then replays it backwards, producing all partial derivatives of a single scalar output in one pass. That makes it the right tool for gradients of many-input, one-output functions such as loss functions and objective functions.

§How it works

  • GradientTape owns a Wengert list: an append-only sequence of nodes, each holding one primitive Op and the primal value it produced.
  • Variable is a Copy handle to a node — its index, its value, and a back-pointer to the tape. Arithmetic operators and math methods evaluate the primal value, push a node, and return a handle to it.
  • Op::diff supplies the local derivative rule for each primitive as a small list of (operand, ∂out/∂operand) Partials.
  • GradientTape::backward seeds the output node with an adjoint of 1 and scans the tape in reverse. Because operands are always recorded before the nodes that consume them, reverse index order is a reverse topological order, so each node’s adjoint is complete by the time it is visited. Contributions are accumulated, so re-using a subexpression (a diamond in the DAG) is handled automatically.

§Examples

use tpt_math_autodiff_rev::GradientTape;

let tape = GradientTape::new();
let x = tape.var(2.0);
let y = tape.var(3.0);
let z = x * x + y.sin();
let g = tape.gradient(z, &[x, y]);

assert!((g[0] - 4.0).abs() < 1e-9); // dz/dx = 2x
assert!((g[1] - 3.0_f64.cos()).abs() < 1e-9); // dz/dy = cos(y)

Supported primitives: +, -, *, / (between variables and with plain f64 on either side), unary -, sin, cos, tan, exp, ln, sqrt, powi, powf and recip, plus constants that never carry a gradient.

Modules§

prelude
Everything needed to build and differentiate an expression.

Structs§

Gradient
The adjoints produced by a backward pass: ∂output/∂node for every node on the tape.
GradientTape
A tape that records every operation performed on its Variables so that all partial derivatives can be recovered with a single backward pass.
Partials
A local Jacobian row: up to MAX_OPERANDS (operand index, derivative) pairs.
Variable
A scalar bound to a GradientTape.

Enums§

Op
A single primitive operation in the tape’s Wengert list.

Constants§

MAX_OPERANDS
The maximum number of operands any Op can reference.

Functions§

value_and_gradient
Evaluate f at inputs and return (value, gradient).