tidu 0.2.0

Automatic-differentiation transforms (linearize, transpose, eager reverse-mode) for primitive computation graphs.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# Complex AD

`tidu` follows the JAX convention for complex automatic differentiation.

Forward mode computes the full real-linear derivative:

```text
df = (df/dz) * dz + (df/dconj(z)) * conj(dz)
```

Reverse mode transposes linear maps with respect to the real inner product:

```text
<a, b> = Re(conj(a) * b)
```

For a general function `f: C -> C`, a cotangent is:

```text
ct_z = ct_y * conj(df/dz) + conj(ct_y) * (df/dconj(z))
```

For real losses, this gives `ct_z = 2 * dL/dconj(z)` when the output cotangent
seed is `1`. That differs from frameworks that directly return `dL/dconj(z)`,
but the steepest-descent direction is the same.