Automatic Differentiation Library

AUTOmatic Derivatives & Jacobians by djmaxus and you

• Single variables

  use autodj::single::*;
  
  let x : DualNumber = 2.0.into_variable();
  
  let f = x * x + &1.0.into(); // can be borrowed for arithmetic operations
  
  assert_eq!(f.value(), 5.0);
  assert_eq!(f.deriv(), 4.0);
  assert_eq!(format!("{f}"), "5+4∆"); // fmt::Display resembles Taylor expansion
  

• Multiple variables They are based on multiple dual components and don't require 'backward' differentiation to be efficient since each partial derivative is tracked separately from the start

  • Static number of variables

    use autodj::array::*;
    
    let vars : DualVariables<2> = [2.0, 3.0].into_variables(); // consistent set of independent variables
    let [x, y] = vars.get().to_owned();
    
    let f = x * (y - 1.0.into());
    
    assert_eq!(f.value(), 4.);
    assert_eq!(f.grad() , &[2., 2.]);
    assert_eq!(format!("{f}"), "4+[2.0, 2.0]∆");
    

  • Dynamic number of variables

    use autodj::vector::*;
    
    let x : DualVariables = vec![1., 2., 3., 4., 5.].into_variables();
    
    let f : DualNumber = x.get().iter().map(|x : &DualNumber| x * &2.0.into()).sum();
    
    assert_eq!(f.value(), 30.);
    f.grad().iter().for_each(|deriv| assert_eq!(deriv, &2.0) );
    

• Generic implementation

  use autodj::common::DualCommon; // can be specialized for your needs
  

Contents

• Contents
• Motivation
• Project goals
• Anticipated features
• Comparison with autodiff

Motivation

I do both academic & business R&D in the area of computational mathematics. As well as many of us, I've written a whole bunch of sophisticated Jacobians by hand.

One day, I learned about automatic differentiation based on dual numbers. Almost the same day, I learned about Rust as well :crab:

Then, I decided to:

• Make it automatic and reliable as much as possible
• Use modern and convenient ecosystem of Rust development

Project goals

• Develop open-source automatic differentiation library for both academic and commercial computational mathematitians
• Gain experience of Rust programming

Anticipated features

You are very welcome to introduce issues to promote most wanted features or to report a bug.

• Generic implementation of dual numbers
• Number of variables to differentiate
  • single
  • multiple
    • static
    • dynamic
    • sparse
  • Jacobians for efficient layouts in memory
• Named variables (UUID-based)
• Calculation tracking (partial derivatives of intermediate values)
• Third-party crates support (as features)
  • num
  • linear algebra crates (nalgebra etc.)
• Advanced features
  • Arbitrary number types beside f64
  • Inter-operability of different dual types (e.g., single and multiple dynamic)
  • Numerical verification (or replacement) of derivatives (by definition)
  • Macro for automatic extensions of regular (i.e. non-dual) functions
  • Optional calculation of derivatives
    • Iterator implementation as possible approach to lazy evaluation

Comparison with autodiff

As far as I noticed, autodj currently has the following differences

• Multiple variables out of the box
• fmt::Display for statically-known number of variables
• Left-to-right flow of many operations such as .into-variables(), .eval(), etc.
• Number type is restricted to f64
• No utilization of num and nalgebra crates

Some defferences are planned to be eliminated as noted in the roadmap.

Within this crate, you may study & launch test target /tests/autodiff.rs to follow some differences.

cargo test --test autodiff -- --show-output