Module autograd 

Reverse-mode automatic differentiation engine for neural network training.

This module implements tape-based automatic differentiation following the methodology described in Baydin et al. (2018) and Griewank & Walther (2008).

Architecture

The autograd engine uses a define-by-run (dynamic) computational graph:

• Operations are recorded to a tape during forward pass
• Gradients are computed in reverse order during backward pass
• Supports gradient accumulation for multi-use tensors

Example

use aprender::autograd::{Tensor, no_grad};

// Create tensors with gradient tracking
let x = Tensor::from_slice(&[1.0, 2.0, 3.0]).requires_grad();
let w = Tensor::from_slice(&[0.5, 0.5, 0.5]).requires_grad();

// Forward pass (operations recorded to tape)
let y = x.mul(&w).sum();

// Backward pass (compute gradients)
y.backward();

// Access gradients
println!("dL/dx = {:?}", x.grad());
println!("dL/dw = {:?}", w.grad());

References

• Baydin, A. G., et al. (2018). Automatic differentiation in machine learning: a survey. JMLR.
• Rumelhart, D. E., et al. (1986). Learning representations by back-propagating errors. Nature.
• Griewank, A., & Walther, A. (2008). Evaluating derivatives. SIAM.

Structs

ComputationGraph

Computation graph that records operations for backward pass.

Tensor

A tensor with optional gradient tracking for automatic differentiation.

TensorId

Unique identifier for tensors in the computation graph.

Traits

GradFn

Trait for functions that compute gradients during backward pass.

Functions

clear_grad

Clear gradient for a specific tensor by ID.

clear_graph

Clear the computation graph (called after backward).

get_grad

Get gradient for a tensor by ID from the graph.

is_grad_enabled

Check if gradient tracking is currently enabled.

no_grad

Execute a closure without gradient tracking.