Expand description
The AAD tape (Wengert list) and the backward sweep.
Every arithmetic operation on Var records one
node holding the indices of its (up to two) parents and the local
partial derivatives with respect to them. The backward sweep walks
the tape once in reverse, accumulating adjoints — so the gradient of
one scalar output with respect to every input costs one forward
evaluation plus one reverse pass, independent of the number of
inputs. That O(1) property is the whole point for Greeks.