Fwd:AD a crate for Forward Auto-Differentiation
This crate allows you to easily write operations on dual numbers and do forward automatic differentiation. It empowers its user to write auto-differentiation code with minimal allocations.
Key selling-points
- Clone-free by default. Fwd:AD will never clone memory in its functions (except
to_owning()
) andstd::ops
implementations, leveraging Rust's ownership system to ensure correctness memory-wise, and leaving it up to the user to be explicit as to when cloning should happen. - Automatic cloning on demand. If passed the
implicit-clone
feature, Fwd:AD will implicitly cloneDual
s when needed. Deciding whether to clone or not is entirely done via the type-system, and hence at compile time. - Generic in memory location: Fwd:AD's structs are generic over a container type, allowing them to be backed by any container of your choice:
Vec
to rely on the heap, arrays if you're more of a stack-person, or other. For example, it can be used with&mut [f64]
to allow an FFI API that won't need to copy memory at its frontier.
Examples
Detailled examples are available in the examples/
directory, but some snippets are reproduced below.
Rosenbrock function minimization
extern crate fwd_ad;
use *;
// The factor by which we will descend along the gradient.
// Rosenbrock function is pretty steep so its quite small.
const ALPHA : f64 = 1e-3;
Short tutorial
Fdw:AD's main type is the Dual<Container, OM, F>
struct. This struct is parametrized by three types, which are:
Container
a type indicating what "container" is used to store the struct content. Typical examples includeVec<F>
,[F; n]
,&mut [F]
, or&[F]
.OM
an "owning mode" which is one of two possibilities:RW
for "read-write", indicating that the content of the Dual is write-able and hence can be reused during computations andRO
indicating that it is read-only.F
is the scalar type, typicallyf32
orf64
, but you chan choose to use something different.
A Dual
wraps its container, which must be "read-able as an [F]
". The first item of this slice of scalars is the dual's actual value and the next ones are the derivative with respect to the successive variables.
To alleviate the burden of writting out long type names, canonical pairs of owning/view duals are defined in the instanciations
module.
Fwd:AD Traits
Fwd:AD relies on several traits to be generic enough. Traits a user may need to implement are located in the traits
module.
ROAble
(resp.RWAble
) are traits that should be implemented by containers which are able to read (resp. write) their content. All container types must implementROAble
. These traits are similar toAsRef
/AsMut
fromcore
and a blanket implementation is provided.ToView
andToOwning
are traits that are used to defined correspondances of canonical "owning" (which can beRW
) and "view" (which only haveRO
capacity) containers.Scalar
is the trait representing scalar numbers, it is merely a supertrait for various traits ofnum_traits
, so these are what you should seek to implement.
Caveat: because you can't implement external traits on external types you may find yourself limited in using duals with an uncommon container or scalar type. If so, please contact the maintainer of this crate.
Comparision with other (forward) AD rust libraries
The last-update column represent the last time the corresponding crate was checked. Crates may have evolved since.
crate | version | multi-variate | higher-order | last update |
---|---|---|---|---|
Fwd:AD | 0.1.0 | ✔️ | ❌ | 2020-04-29 |
ad | 0.1.0 | ❌ | ❌ | 2020-01-01 |
autodiff | 0.1.9 | ❌ | ❌ | 2019-11-07 |
descent¹ | 0.3 | ✔️ | (2nd order?) | 2018-12-10 |
dual | 0.2.0 | ❌ | ❌ | 2015-12-25 |
dual_num | 0.2.7 | ❌ | ❌ | 2019-04-03 |
hyperdual² | 0.3.4 | ✔️ | ❌ | 2020-02-08 |
peroxide | 0.21.7 | ❌ | (2nd order) | 2020-04-21 |
descent
Automatic differentiation seems promising but isn't very documented and is mixed-up with the IP-OPT interfacehyperdual
has similar properties to Fwd:AD, except that all operations will allocate when Fwd:AD tries to reuse existing memory