autodj 0.4.2

Automatic Differentiation Library
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132

# Automatic Differentiation Library


[![crates.io](https://img.shields.io/crates/v/autodj.svg)](https://crates.io/crates/autodj)
[![docs](https://docs.rs/autodj/badge.svg)](https://docs.rs/autodj/)
![build](https://github.com/djmaxus/autodj/actions/workflows/rust.yml/badge.svg?branch=master)
![rust-clippy analyze](https://github.com/djmaxus/autodj/actions/workflows/rust-clippy.yml/badge.svg?branch=master)

**AUTO**matic **D**erivatives & **J**acobians
by [djmaxus](https://djmaxus.github.io/) and [you](https://github.com/djmaxus/autodj/issues)

- [x] **Single variables**

  ```rust
  use autodj::single::*;

  let x : DualNumber = 2.0.into_variable();

  let f = x * x + &1.0.into(); // can be borrowed for arithmetic operations

  assert_eq!(f.value(), 5.0);
  assert_eq!(f.deriv(), 4.0);
  assert_eq!(format!("{f}"), "5+4∆"); // fmt::Display resembles Taylor expansion
  ```

- [x] **Multiple variables**
      They are based on **multiple dual components**
      and **don't require 'backward' differentiation** to be efficient
      since each partial derivative is tracked separately from the start

  - **Static** number of variables

    ```rust
    use autodj::array::*;

    let vars : DualVariables<2> = [2.0, 3.0].into_variables(); // consistent set of independent variables
    let [x, y] = vars.get().to_owned();

    let f = x * (y - 1.0.into());

    assert_eq!(f.value(), 4.);
    assert_eq!(f.grad() , &[2., 2.]);
    assert_eq!(format!("{f}"), "4+[2.0, 2.0]∆");
    ```

  - **Dynamic** number of variables

    ```rust
    use autodj::vector::*;

    let x : DualVariables = vec![1., 2., 3., 4., 5.].into_variables();

    let f : DualNumber = x.get().iter().map(|x : &DualNumber| x * &2.0.into()).sum();

    assert_eq!(f.value(), 30.);
    f.grad().iter().for_each(|deriv| assert_eq!(deriv, &2.0) );
    ```

- [x] **Generic implementation**

  ```rust
  use autodj::common::DualCommon; // can be specialized for your needs
  ```

## Contents


- [Contents](#contents)
- [Motivation](#motivation)
- [Project goals](#project-goals)
- [Anticipated features](#anticipated-features)
- [Comparison with `autodiff`](#comparison-with-autodiff)

## Motivation


I do both academic & business R&D in the area of computational mathematics.
As well as many of us, I've written a whole bunch of sophisticated Jacobians _by hand_.

One day, I learned about automatic differentiation based on dual numbers.
Almost the same day, I learned about Rust as well :crab:

Then, I decided to:

- Make it automatic and reliable as much as possible
- Use modern and convenient ecosystem of Rust development

## Project goals


- Develop open-source automatic differentiation library for both _academic_ and _commercial_ computational mathematitians
- Gain experience of Rust programming

## Anticipated features


You are very welcome to introduce [issues](https://github.com/djmaxus/autodj/issues/new/choose) to promote most wanted features or to report a bug.

- [x] Generic implementation of dual numbers
- Number of variables to differentiate
  - [x] single
  - multiple
    - [x] static
    - [x] dynamic
    - [ ] sparse
  - [ ] Jacobians for efficient layouts in memory
- [ ] Named variables (UUID-based)
- [ ] Calculation tracking (partial derivatives of intermediate values)
- Third-party crates support (as features)
  - [ ] `num`
  - [ ] linear algebra crates (`nalgebra` etc.)
- Advanced features
  - [ ] Arbitrary number types beside `f64`
  - [ ] Inter-operability of different dual types (e.g., single and multiple dynamic)
  - [ ] Numerical verification (or replacement) of derivatives (by definition)
  - [ ] Macro for automatic extensions of regular (i.e. non-dual) functions
  - [ ] Optional calculation of derivatives
    - [ ] Iterator implementation as possible approach to lazy evaluation

## Comparison with [`autodiff`](https://crates.io/crates/autodiff)


As far as I noticed, `autodj` currently has the following differences

- Multiple variables out of the box
- `fmt::Display` for statically-known number of variables
- Left-to-right flow of many operations such as `.into-variables()`, `.eval()`, etc.
- Number type is restricted to `f64`
- No utilization of `num` and `nalgebra` crates

Some defferences are planned to be eliminated as noted in the [roadmap](#anticipated-features).

Within this crate, you may study & launch test target `/tests/autodiff.rs`
to follow some differences.

```shell
cargo test --test autodiff -- --show-output
```