Module reverse_mode 

Reverse-mode automatic differentiation (backpropagation)

Reverse-mode AD is efficient for computing derivatives when the number of output variables is small (typically 1 for optimization). It builds a computational graph and then propagates derivatives backwards.

Structs

ComputationGraph

Computational graph for reverse-mode AD

ReverseADOptions

Options for reverse-mode automatic differentiation

ReverseVariable

Variable in the computational graph for reverse-mode AD

Functions

abs

Absolute value operation on computation graph

add

Addition operation on computation graph

cos

Cosine operation on computation graph

div

Division operation on computation graph

exp

Exponential operation on computation graph

is_reverse_mode_efficient

Check if reverse mode is preferred for the given problem dimensions

leaky_relu

Leaky ReLU operation on computation graph

ln

Natural logarithm operation on computation graph

mul

Multiplication operation on computation graph

powi

Power operation (x^n) on computation graph

relu

ReLU (Rectified Linear Unit) operation on computation graph

reverse_gauss_newton_hessian

Gauss-Newton Hessian approximation using reverse-mode AD

reverse_gauss_newton_hessian_ad

Gauss-Newton Hessian approximation using reverse-mode AD for AD-compatible functions

reverse_gradient

Compute gradient using reverse-mode automatic differentiation This is a generic function that works with closures, using finite differences For functions that can be expressed in terms of AD operations, use reverse_gradient_with_tape

reverse_gradient_ad

Compute gradient using reverse-mode AD with a function that directly uses AD operations

reverse_gradient_with_tape

Simple reverse-mode gradient computation using a basic tape

reverse_hessian

Compute Hessian using reverse-mode automatic differentiation (finite differences for generic functions)

reverse_hessian_ad

Compute Hessian using forward-over-reverse mode for AD functions

reverse_vjp

Vector-Jacobian product using reverse-mode AD

reverse_vjp_ad

Vector-Jacobian product using reverse-mode AD for AD-compatible functions

sigmoid

Sigmoid operation on computation graph

sin

Sine operation on computation graph

sqrt

Square root operation on computation graph

sub

Subtraction operation on computation graph

tan

Tangent operation on computation graph

tanh

Hyperbolic tangent operation on computation graph