pub struct Record<'a, T: Primitive> {
    pub number: T,
    pub index: usize,
    /* private fields */
}
Expand description

A wrapper around a real number which records it going through the computational graph. This is used to perform Reverse Mode Automatic Differentiation.

Panics

Every operation and nearly every method a Record has involves manipulating the record’s history on its referenced WengertList. This WengertList itself maintains a RefCell which tracks borrows at runtime rather than compile time. This is neccessary to maintain the illusion that Records are just ordinary numbers, and the side effects of doing arithmetic with Records are limited to their referenced WengertList. Hence, the Rust compiler infers that it is not safe to share references to WengertLists between threads, nor transfer Records across threads. If you called a method on two Records that both mutably borrowed from the same WengertList at once, which could be trivially done with multiple threads, then the code would panic. Easy ML shouldn’t allow you to do this in safe Rust because each mutable borrow of the WengertList is dropped at the end of each Record method call, and you can’t call two methods simulatenously without threading.

Acknowledgments

A tutorial by Rufflewind and the associated MIT licensed soure code were invaluable in providing understanding on how to implement Reverse Mode Automatic Differentiation.

Fields

number: T

The real number

index: usize

The index of this number in its WengertList. The first entry will be 0, the next 1 and so on.

In normal use cases you should not need to read this field, you can index Derivatives directly with Records.

Implementations

The main set of methods for using Record types for Reverse Differentiation.

The general steps are

  1. create a WengertList
  2. create variables from this list
  3. do operations on the variables
  4. from the output you want to compute derivatives for call .derivatives()
  5. index the Derivatives object with the index variables to get the derivatives with respect to each input
  6. if you want to make another pass call clear() on the WengertList and then call reset() on all of the variables to forget the gradients already computed (the order of clear then reset is very important!).

Constants can be used to save memory if you have numbers that you do not need to compute the gradients with respect to.

Creates an untracked Record which has no backing WengertList.

This is provided for using constants along with Records in operations.

For example with y = x + 4 the computation graph could be conceived as a y node with parent nodes of x and 4 combined with the operation +. However there is no need to record the derivatives of a constant, so instead the computation graph can be conceived as a y node with a single parent node of x and the unary operation of +4.

This is also used for the type level constructors required by Numeric which are also considered constants.

Creates a record backed by the provided WengertList.

The record cannot live longer than the WengertList, hence the following example does not compile

use easy_ml::differentiation::Record;
use easy_ml::differentiation::WengertList;
let record = {
    let list = WengertList::new();
    Record::variable(1.0, &list)
}; // list no longer in scope

You can alternatively use the record constructor on the WengertList type.

Resets this Record to place it back on its WengertList, for use in performing another derivation after clearing the WengertList.

A convenience helper function which takes a Record by value and calls reset on it.

Performs a backward pass up this record’s WengertList from this record as the output, computing all the derivatives for the inputs involving this output.

If you have N inputs x1 to xN, and this output is y, then this computes all the derivatives δy/δxi for i = 1 to N.

Panics

Panics if the Record has no backing WengertList, ie it was created as a constant.

Creates a new Record from a reference to an existing Record by applying some unary function to it which operates on the type the Record wraps.

To compute the new record, the unary function of some input x to some output y is needed along with its derivative with respect to its input x.

For example, tanh is a commonly used activation function, but the Real trait does not include this operation and Record has no operations for it specifically. However, you can use this function to compute the tanh of a Record like so:

use easy_ml::differentiation::{Record, WengertList};
let list = WengertList::new();
let x = Record::variable(0.7f32, &list);
// the derivative of tanh(x) is sech(x) * sech(x) which is equivalent to
// 1 / (cosh(x) * cosh(x))
let y = x.unary(|x| x.tanh(), |x| 1.0 / (x.cosh() * x.cosh()));
assert_eq!(y.derivatives()[&x], 1.0f32 / (0.7f32.cosh() * 0.7f32.cosh()));

Creates a new Record from a reference to two existing Records by applying some binary function to them which operates on two arguments of the type the Records wrap.

To compute the new record, the binary function of some inputs x and y to some output z is needed along with its derivative with respect to its first input x and its derivative with respect to its second input y.

For example, atan2 takes two arguments, but the Real trait does not include this operation and Record has no operations for it specifically. However, you can use this function to compute the atan2 of two Records like so:

use easy_ml::differentiation::{Record, WengertList};
let list = WengertList::new();
let x = Record::variable(3.0f32, &list);
let y = Record::variable(3.0f32, &list);
// the derivative of atan2 with respect to x is y/(x*x + y*y)
// https://www.wolframalpha.com/input/?i=d%28atan2%28x%2Cy%29%29%2Fdx
// the derivative of atan2 with respect to y is -x/(x*x + y*y)
// https://www.wolframalpha.com/input/?i=d%28atan2%28x%2Cy%29%29%2Fdy
let z = x.binary(&y,
    |x, y| x.atan2(y),
    |x, y| y/((x*x) + (y*y)),
    |x, y| -x/((x*x) + (y*y))
);
let derivatives = z.derivatives();
let dx = derivatives[&x];
let dy = derivatives[&y];

Trait Implementations

Operation for two records of the same type with the right referenced.

The resulting type after applying the + operator.

Performs the + operation. Read more

Addition for a record and a constant of the same type with both referenced.

The resulting type after applying the + operator.

Performs the + operation. Read more

Operation for a record and a constant of the same type with the right referenced.

The resulting type after applying the + operator.

Performs the + operation. Read more

Addition for two records of the same type with both referenced and both using the same WengertList.

The resulting type after applying the + operator.

Performs the + operation. Read more

Operation for two records of the same type.

The resulting type after applying the + operator.

Performs the + operation. Read more

Operation for two records of the same type with the left referenced.

The resulting type after applying the + operator.

Performs the + operation. Read more

Operation for a record and a constant of the same type.

The resulting type after applying the + operator.

Performs the + operation. Read more

Operation for a record and a constant of the same type with the left referenced.

The resulting type after applying the + operator.

Performs the + operation. Read more

Any record of a Cloneable type implements clone

Returns a copy of the value. Read more

Performs copy-assignment from source. Read more

Cosine of a Record by reference.

Operation for a record by value.

Formats the value using the given formatter. Read more

A record is displayed by showing its number component.

Formats the value using the given formatter. Read more

Operation for two records of the same type with the right referenced.

The resulting type after applying the / operator.

Performs the / operation. Read more

Division for a record and a constant of the same type with both referenced.

The resulting type after applying the / operator.

Performs the / operation. Read more

Operation for a record and a constant of the same type with the right referenced.

The resulting type after applying the / operator.

Performs the / operation. Read more

Dvision for two records of the same type with both referenced and both using the same WengertList.

The resulting type after applying the / operator.

Performs the / operation. Read more

Operation for two records of the same type.

The resulting type after applying the / operator.

Performs the / operation. Read more

Operation for two records of the same type with the left referenced.

The resulting type after applying the / operator.

Performs the / operation. Read more

Operation for a record and a constant of the same type.

The resulting type after applying the / operator.

Performs the / operation. Read more

Operation for a record and a constant of the same type with the left referenced.

The resulting type after applying the / operator.

Performs the / operation. Read more

Exponential, ie ex of a Record by reference.

Operation for a record by value.

Quries the derivative at the provided record as input.

If you construct a Derivatives object for some output y, and call .at(&x) on it for some input x, this returns dy/dx.

The returned type after indexing.

Natural logarithm, ie ln(x) of a Record by reference.

Operation for a record by value.

Operation for two records of the same type with the right referenced.

The resulting type after applying the * operator.

Performs the * operation. Read more

Multiplication for a record and a constant of the same type with both referenced.

The resulting type after applying the * operator.

Performs the * operation. Read more

Operation for a record and a constant of the same type with the right referenced.

The resulting type after applying the * operator.

Performs the * operation. Read more

Multiplication for two records of the same type with both referenced and both using the same WengertList.

The resulting type after applying the * operator.

Performs the * operation. Read more

Operation for two records of the same type.

The resulting type after applying the * operator.

Performs the * operation. Read more

Operation for two records of the same type with the left referenced.

The resulting type after applying the * operator.

Performs the * operation. Read more

Operation for a record and a constant of the same type.

The resulting type after applying the * operator.

Performs the * operation. Read more

Operation for a record and a constant of the same type with the left referenced.

The resulting type after applying the * operator.

Performs the * operation. Read more

Negation of a record by reference.

The resulting type after applying the - operator.

Performs the unary - operation. Read more

Negation of a record by value.

The resulting type after applying the - operator.

Performs the unary - operation. Read more

Any record of a PartialEq type implements PartialEq

Note that as a Record is intended to be substitutable with its type T only the number parts of the record are compared.

This method tests for self and other values to be equal, and is used by ==. Read more

This method tests for !=.

Any record of a PartialOrd type implements PartialOrd

Note that as a Record is intended to be substitutable with its type T only the number parts of the record are compared.

This method returns an ordering between self and other values if one exists. Read more

This method tests less than (for self and other) and is used by the < operator. Read more

This method tests less than or equal to (for self and other) and is used by the <= operator. Read more

This method tests greater than (for self and other) and is used by the > operator. Read more

This method tests greater than or equal to (for self and other) and is used by the >= operator. Read more

Operation for two records of the same type with the right referenced.

Power of a constant to a Record of the same type with both referenced.

Operation for a constant and a record of the same type with the right referenced.

Power of one Record to a constant of the same type with both referenced.

Operation for a record and a constant of the same type with the right referenced.

Power of one Record to another, ie self^rhs for two records of the same type with both referenced and both using the same WengertList.

Operation for two records of the same type.

Operation for two records of the same type with the left referenced.

Operation for a constant and a record of the same type.

Operation for a constant and a record of the same type with the left referenced.

Operation for a record and a constant of the same type.

Operation for a record and a constant of the same type with the left referenced.

Sine of a Record by reference.

Operation for a record by value.

Square root of a Record by reference.

Operation for a record by value.

Operation for two records of the same type with the right referenced.

The resulting type after applying the - operator.

Performs the - operation. Read more

Subtraction for a record and a constant of the same type with both referenced.

The resulting type after applying the - operator.

Performs the - operation. Read more

Operation for a record and a constant of the same type with the right referenced.

The resulting type after applying the - operator.

Performs the - operation. Read more

Subtraction for two records of the same type with both referenced and both using the same WengertList.

The resulting type after applying the - operator.

Performs the - operation. Read more

Operation for two records of the same type.

The resulting type after applying the - operator.

Performs the - operation. Read more

Operation for two records of the same type with the left referenced.

The resulting type after applying the - operator.

Performs the - operation. Read more

Operation for a record and a constant of the same type.

The resulting type after applying the - operator.

Performs the - operation. Read more

Operation for a record and a constant of the same type with the left referenced.

The resulting type after applying the - operator.

Performs the - operation. Read more

Any record of a Numeric type implements Sum, which is the same as adding a bunch of Record types together.

Method which takes an iterator and generates Self from the elements by “summing up” the items. Read more

Subtraction for a record and a constant, where the constant is the left hand side, ie C - record.

Division for a record and a constant, where the constant is the left hand side, ie C / record.

Subtraction for a record and a constant, where the constant is the left hand side, ie C - record.

Division for a record and a constant, where the constant is the left hand side, ie C / record.

Subtraction for a record and a constant, where the constant is the left hand side, ie C - record.

Division for a record and a constant, where the constant is the left hand side, ie C / record.

Subtraction for a record and a constant, where the constant is the left hand side, ie C - record.

Division for a record and a constant, where the constant is the left hand side, ie C / record.

Any record of a Copy type implements Copy

Auto Trait Implementations

Blanket Implementations

Gets the TypeId of self. Read more

Immutably borrows from an owned value. Read more

Mutably borrows from an owned value. Read more

Returns the argument unchanged.

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

The resulting type after obtaining ownership.

Creates owned data from borrowed data, usually by cloning. Read more

🔬 This is a nightly-only experimental API. (toowned_clone_into)

Uses borrowed data to replace owned data, usually by cloning. Read more

Converts the given value to a String. Read more

The type returned in the event of a conversion error.

Performs the conversion.

The type returned in the event of a conversion error.

Performs the conversion.