1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
// SPDX-License-Identifier: Apache-2.0 OR MIT
//! Append-only Wengert list (tape) for recording the computational graph.
//!
//! Operations are recorded in forward-execution order via [`Tape::push`].
//! During backward, the tape is traversed using Kahn's algorithm (built
//! in a subsequent chunk) with the edge counts stored in
//! [`TensorMeta::total_grads`](crate::tensor::TensorMeta::total_grads).
use crateBackwardOp;
use crate;
// ---------------------------------------------------------------------------
// TapeEntry
// ---------------------------------------------------------------------------
/// A single node in the computational graph.
///
/// Records which backward operation produced which outputs from which
/// inputs, along with the [`BackwardOp`] that knows how to compute
/// gradients.
///
/// # Edge semantics
///
/// - `inputs` lists the [`GradId`]s of the tensors this op **consumed**.
/// During backward, computed gradients are propagated **to** these ids.
///
/// - `outputs` lists the [`GradId`]s of the tensors this op **produced**.
/// During backward, the incoming gradient is read **from** these ids.
///
/// Per Tenet #3 ("Strict Graph Edge Counting"): if a tensor is used twice
/// as an input to a single op, its `GradId` appears twice in `inputs`,
/// and its `TensorMeta::total_grads` was incremented by two at record
/// time.
// ---------------------------------------------------------------------------
// Tape
// ---------------------------------------------------------------------------
/// Append-only Wengert list of recorded operations.
///
/// The tape is the backbone of the autograd engine. During the forward
/// pass, every differentiable operation appends a [`TapeEntry`]. During
/// backward, the engine reads entries (by [`OpId`] index) to determine
/// what gradient math to run and where to propagate the results.
///
/// The tape is intentionally **not** thread-safe — it is owned by a single
/// forward-pass context. If we need parallel forward passes in the future,
/// each will own its own tape.