Struct neuronika::VarDiff

pub struct VarDiff<T, U> where
    T: Data + 'static,
    U: Gradient + Overwrite + 'static,  { /* fields omitted */ }

A differentiable variable.

Differentiable variables can be created in the two following ways described hereafter:

1. By calling .requires_grad() on a non-differentiable leaf.

2. By performing any binary operation between a Var and a VarDiff. Differentiability is thus a contagious property, that is, if during a computation a VarDiff is used, the result of the computation itself, and that of any other subsequent computations performed on it, will also be differentiable. As an obvious consequence, the results of operations performed on two VarDiff will also be VarDiff.

Implementations

impl<T, U> VarDiff<T, U> where
    T: Data + 'static,
    U: Gradient + Overwrite + 'static, 

pub fn data(&self) -> Ref<'_, Array<f32, T::Dim>>

Returns an immutable reference to the data inside self.

At the differentiable variable’s creation the data is filled with zeros. You can populate it with a call to .forward().

pub fn data_mut(&self) -> RefMut<'_, Array<f32, T::Dim>>

Returns a mutable reference to the data inside self.

At the differentiable variable’s creation the data is filled with zeros. You can populate it with a call to .forward().

pub fn grad(&self) -> Ref<'_, Array<f32, U::Dim>>

Returns an immutable reference to the gradient inside self.

At the differentiable variable’s creation the gradient is filled with zeros. You can populate it with a call to .backward().

pub fn grad_mut(&self) -> RefMut<'_, Array<f32, U::Dim>>

Returns a mutable reference to the gradient inside self.

At the differentiable variable’s creation the gradient is filled with zeros. You can populate it with a call to .backward().

impl<T, U> VarDiff<T, U> where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static, 

pub fn forward(&self)

Propagates the computations forwards and populates all the variables and differentiable variables from the leaves of the graph to self.

impl<T, U> VarDiff<T, U> where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + Backward + 'static, 

pub fn backward(&self, seed: f32)

Back-propagates through the computational graph and populates the gradients of the differentiable leaves that are ancestors of self. Before back-propagating the gradient of self is seeded with seed, thus, the leaves’ gradients will be scaled accordingly.

The graph is differentiated through the chain rule.

The leaves whose gradients are populated by this method are also those referred by the vector of Param returned by .parameters().

pub fn no_grad(&self)

Disables gradient computation and de-allocates the gradient for self and all of its ancestors.

pub fn with_grad(&self)

Re-enables gradient computation and re-allocates the gradient for self and all of its ancestors.

pub fn train(&self)

This has effect only on certain ancestor variables of self. It sets such variables and differentiable variables in training mode.

See also .dropout().

pub fn eval(&self)

This has effect only on certain ancestor variables of self. It sets such variables and differentiable variables in evaluation mode.

See also .dropout().

impl<T, U> VarDiff<T, U> where
    T: Data<Dim = Ix1> + 'static,
    U: Gradient<Dim = Ix1> + Overwrite + 'static, 

pub fn vm<Rhs>(self, rhs: Rhs) -> <Self as VecMatMul<Rhs>>::Output where
    Self: VecMatMul<Rhs>, 

Performs a vector-matrix multiplication between the vector variable self and the matrix variable rhs.

If self is n and rhs is (n, m) the output will be m.

pub fn vv<Rhs>(self, rhs: Rhs) -> <Self as VecVecMul<Rhs>>::Output where
    Self: VecVecMul<Rhs>, 

Vector-vector product, a.k.a. scalar product or inner product.

Performs the scalar product between the two vector variables self and rhs.

impl<T, U> VarDiff<T, U> where
    T: Data<Dim = Ix2> + 'static,
    U: Gradient<Dim = Ix2> + Overwrite + 'static, 

pub fn mm<Rhs>(self, rhs: Rhs) -> <Self as MatMatMul<Rhs>>::Output where
    Self: MatMatMul<Rhs>, 

Performs a matrix multiplication between the matrix variables self and rhs. If self is (n, m) and rhs is (m, o) the output will be (n, o).

pub fn mm_t<Rhs>(self, rhs: Rhs) -> <Self as MatMatMulT<Rhs>>::Output where
    Self: MatMatMulT<Rhs>, 

Performs a matrix multiplication between the matrix variables self and rhs. This is a fused operation as rhs is implicitly transposed. Fusing the two operations it’s marginally faster than computing the matrix multiplication and the transposition separately.

If self is (n, m) and rhs is (o, m) the output will be (n, o).

pub fn mv<Rhs>(self, rhs: Rhs) -> <Self as MatVecMul<Rhs>>::Output where
    Self: MatVecMul<Rhs>, 

Performs a matrix-vector multiplication between the matrix variable self and the vector variable rhs.

If self is (n, m) and rhs is m the output will be n.

impl<T, U> VarDiff<T, U> where
    T: Data + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static, 

pub fn parameters(&self) -> Vec<Param<'_>>

Returns a vector of Param referencing all the differentiable leaves that are ancestors of the variable.

If directly called on a differentiable leaf the resulting vector will include only a single Param referencing self.

Ancestors that appear multiple times in the computation of the variable are listed only once. Thus, the parameters of a differentiable variable z resulting from a binary operation involving two other differentiable variables x and y will be the set union of the parameters of x and y. This can be extended to the general case.

Examples

use neuronika;

// x has 2 parameters, as it is the result of an addition.
let x = neuronika::rand((3,3)).requires_grad() + neuronika::rand((3,3)).requires_grad();
// The same holds for y.
let y = neuronika::rand(3).requires_grad() + neuronika::rand(1).requires_grad();

assert!(x.parameters().len() == y.parameters().len() && y.parameters().len() == 2);

// z is the result of an addition between x and y, so it will have 4 parameters.
let z = x.clone() + y;
assert_eq!(z.parameters().len(), 4);

// If we add x to z there still will be 4 parameters, as x is already present among them.
let w = z + x;
assert_eq!(w.parameters().len(), 4);

pub fn sum(self) -> VarDiff<Sum<T>, SumBackward<U>>

Returns the sum of all elements in self.

pub fn mean(self) -> VarDiff<Mean<T>, MeanBackward<U>>

Returns the mean of all elements in self.

pub fn pow(self, exp: i32) -> VarDiff<Power<T>, PowerBackward<U, T>>

Takes the power of each element in self with exponent exp and returns a differentiable variable with the result.

pub fn sqrt(self) -> VarDiff<Sqrt<T>, SqrtBackward<U, Sqrt<T>>>

Takes the square root element-wise and returns a differentiable variable with the result.

pub fn relu(self) -> VarDiff<ReLU<T>, ReLUBackward<U, T>>

Applies the rectified linear unit element-wise and and returns a differentiable variable with the result.

ReLU(x) = max(0, x)

pub fn leaky_relu(self) -> VarDiff<LeakyReLU<T>, LeakyReLUBackward<U, T>>

Applies the leaky rectified linear unit element-wise and returns a differentiable variable with the result.

LeakyReLU(x) = max(0, x) + 0.01 * min(0, x)

pub fn softplus(self) -> VarDiff<SoftPlus<T>, SoftPlusBackward<U, T>>

Applies the softplus element-wise and returns a differentiable variable with the result.

Softplus(x) = log(1 + exp(x))

pub fn sigmoid(self) -> VarDiff<Sigmoid<T>, SigmoidBackward<U, Sigmoid<T>>>

Applies the sigmoid element-wise and returns a differentiable variable with the result.

pub fn tanh(self) -> VarDiff<TanH<T>, TanHBackward<U, TanH<T>>>

Applies the tanh element-wise and returns a differentiable variable with the result.

pub fn ln(self) -> VarDiff<Logn<T>, LognBackward<U, T>>

Applies the natural logarithm element-wise and returns a differentiable variable with the result.

pub fn exp(self) -> VarDiff<Exp<T>, ExpBackward<U, Exp<T>>>

Applies the exponential element-wise and returns a differentiable variable with the result.

pub fn softmax(
    self,
    axis: usize
) -> VarDiff<Softmax<T>, SoftmaxBackward<U, Softmax<T>>>

Applies the softmax to self and returns a differentiable variable with the result.

The softmax is applied to all slices along axis, and will re-scale them so that the elements lie in the range [0, 1] and sum to 1.0.

pub fn log_softmax(
    self,
    axis: usize
) -> VarDiff<LogSoftmax<T>, LogSoftmaxBackward<U, LogSoftmax<T>>>

Applies the log-softmax to self and returns a differentiable variable with the result.

Applies a softmax followed by a logarithm. While mathematically equivalent to log(softmax(x)), doing these two operations separately is slower, and numerically unstable. This function uses an alternative formulation to compute the output and gradient correctly.

See also .softmax().

pub fn t(self) -> VarDiff<Transpose<T>, TransposeBackward<U>>

Returns a differentiable variable equivalent to self with its dimensions reversed.

pub fn dropout(self, p: f64) -> VarDiff<Dropout<T>, DropoutBackward<U, T>>

Applies dropout to self and returns a differentiable variable with the result.

It is strongly suggested to use nn::Dropout instead of this method when working with neural networks.

During training, randomly zeroes some of the elements of self with probability p using samples from a Bernoulli distribution. Each channel will be zeroed out independently on every forward call.

This has proven to be an effective technique for regularization and preventing the co-adaptation of neurons as described in the paper Improving neural networks by preventing co-adaptation of feature detectors.

Furthermore, the outputs are scaled by a factor of 1/(1 - p) during training. This means that during evaluation the resulting variable simply computes an identity function.

pub fn chunks<E>(
    self,
    chunk_size: E
) -> Vec<VarDiff<Chunk<T>, ChunkBackward<U>>> where
    E: IntoDimension<Dim = T::Dim>, 

Splits self into a certain number of chunks of size chunk_size skipping the remainder along each dimension that doesn’t fit evenly.

pub fn unsqueeze(
    self,
    axis: usize
) -> VarDiff<Unsqueeze<T>, UnsqueezeBackward<U>>

Returns a new differentiable variable with a dimension of size one inserted at the position specified by axis.

impl<T, U> VarDiff<T, U> where
    T: Data + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    T::Dim: RemoveAxis, 

pub fn cat(
    self,
    variables: &[Box<dyn DifferentiableVariable<T::Dim>>],
    axis: usize
) -> VarDiff<MultiConcatenate<T::Dim>, MultiConcatenateBackward<T::Dim>>

Concatenates the given sequence of differentiable variables variables, including self, along the given axis, and returns a differentiable variable with the results.

Arguments

• variables - sequence of differentiable variables.

• axis - axis to concatenate along to.

Panics

If the variables have mismatching shapes, apart from along axis, if the variables are empty, if axis is out of bounds or if the result is larger than is possible to represent.

Examples

use std::boxed::Box;
use neuronika;
use ndarray;


let a = neuronika::ones((3, 2)).requires_grad();
let b = neuronika::full((3, 2), 4.).requires_grad();
let c = neuronika::full((3, 2), 3.).requires_grad();

let mut d = a.cat(&[Box::new(b), Box::new(c)], 1);
d.forward();

assert_eq!(*d.data(), ndarray::array![[1., 1., 4., 4., 3., 3.],
                                      [1., 1., 4., 4., 3., 3.],
                                      [1., 1., 4., 4., 3., 3.]]);

pub fn stack(
    self,
    variables: &[Box<dyn DifferentiableVariable<T::Dim>>],
    axis: usize
) -> VarDiff<MultiStack<T::Dim>, MultiStackBackward<T::Dim>>

Stacks the given sequence of differentiable variables variables, including self, along the given axis, and returns a differentiable variable with the results.

All variables must have the same shape.

Arguments

• variables - sequence of differentiable variables.

• axis - axis to stack along to.

Panics

If the variables have mismatching shapes, apart from along axis, if the variables are empty, if axis is out of bounds or if the result is larger than is possible to represent.

Examples

use std::boxed::Box;
use neuronika;
use ndarray;


let a = neuronika::ones((2, 2)).requires_grad();
let b = neuronika::ones((2, 2)).requires_grad();
let c = neuronika::ones((2, 2)).requires_grad();

let mut d = a.stack(&[Box::new(b), Box::new(c)], 0);
d.forward();

assert_eq!(*d.data(), ndarray::array![[[1., 1.],
                                       [1., 1.]],
                                      [[1., 1.],
                                       [1., 1.]],
                                      [[1., 1.],
                                       [1., 1.]]]);

Trait Implementations

impl<F1, B1, F2> Add<Var<F2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<F2::Dim>, 

type Output = VarDiff<Addition<F1, F2>, AdditionBackwardUnary<B1, F2>>

The resulting type after applying the + operator.

fn add(self, rhs: Var<F2>) -> Self::Output

Performs the + operation.

impl<F1, F2, B2> Add<VarDiff<F2, B2>> for Var<F1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B2::Dim: Dimension + DimMax<F1::Dim>, 

type Output = VarDiff<Addition<F1, F2>, AdditionBackwardUnary<B2, F1>>

The resulting type after applying the + operator.

fn add(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the + operation.

impl<F1, B1, F2, B2> Add<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<B2::Dim>, 

type Output = VarDiff<Addition<F1, F2>, AdditionBackward<B1, B2>>

The resulting type after applying the + operator.

fn add(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the + operation.

impl<T, U> Add<VarDiff<T, U>> for f32 where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    Ix1: DimMax<T::Dim>,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Addition<Input<Ix1>, T>, AdditionBackwardUnary<U, Input<Ix1>>>

The resulting type after applying the + operator.

fn add(self, rhs: VarDiff<T, U>) -> Self::Output

Performs the + operation.

impl<T, U> Add<f32> for VarDiff<T, U> where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Addition<T, Input<Ix1>>, AdditionBackwardUnary<U, Input<Ix1>>>

The resulting type after applying the + operator.

fn add(self, rhs: f32) -> Self::Output

Performs the + operation.

impl<F1, B1, F2> Cat<Var<F2>> for VarDiff<F1, B1> where
    F1: Data<Dim = B1::Dim> + 'static,
    F2: Data<Dim = F1::Dim> + 'static,
    B1: Gradient + Overwrite + 'static,
    F1::Dim: RemoveAxis,
    B1::Dim: RemoveAxis, 

type Output = VarDiff<Concatenate<F1, F2>, ConcatenateBackwardLeft<B1>>

The type of the concatenation’s result. See the differentiability arithmetic for more details.

fn cat(self, rhs: Var<F2>, axis: usize) -> Self::Output

Concatenates variables along the given axis.

impl<F1, F2, B2> Cat<VarDiff<F2, B2>> for Var<F1> where
    F1: Data<Dim = B2::Dim> + 'static,
    F2: Data<Dim = F1::Dim> + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: RemoveAxis,
    B2::Dim: RemoveAxis, 

type Output = VarDiff<Concatenate<F1, F2>, ConcatenateBackwardRight<B2>>

The type of the concatenation’s result. See the differentiability arithmetic for more details.

fn cat(self, rhs: VarDiff<F2, B2>, axis: usize) -> Self::Output

Concatenates variables along the given axis.

impl<F1, B1, F2, B2> Cat<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    F2: Data<Dim = F1::Dim> + 'static,
    B2: Gradient<Dim = B1::Dim> + Overwrite + 'static,
    F1::Dim: RemoveAxis,
    B1::Dim: RemoveAxis, 

type Output = VarDiff<Concatenate<F1, F2>, ConcatenateBackward<B1, B2>>

The type of the concatenation’s result. See the differentiability arithmetic for more details.

fn cat(self, rhs: VarDiff<F2, B2>, axis: usize) -> Self::Output

Concatenates variables along the given axis.

impl<T, U> Clone for VarDiff<T, U> where
    T: Data + 'static,
    U: Gradient + Overwrite + 'static, 

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<F1, F2, B2, Pad> Convolve<Var<F1>, VarDiff<F2, B2>, Pad> for Var<F1> where
    F1: NData + 'static,
    F1::Dim: RemoveAxis,
    <F1::Dim as Dimension>::Smaller: RemoveAxis,
    <<F1::Dim as Dimension>::Smaller as Dimension>::Smaller: ReflPad + ReplPad,
    F2: NData<Dim = F1::Dim> + 'static,
    B2: Gradient<Dim = F2::Dim> + Overwrite + Display + Debug,
    Pad: PaddingMode + 'static, 

type Output = VarDiff<Convolution<F1, F2, Pad>, ConvolutionBackwardUnary<F1, B2, Pad>>

The type of the convolution’s result. See the differentiability arithmetic for more details.

fn convolve(
    input: Self,
    kernel: VarDiff<F2, B2>,
    stride: &[usize],
    dilation: &[usize],
    padding: &[usize],
    padding_mode: Pad
) -> Self::Output

Applies a n-dimensional convolution with the given parameters. n can be either 1, 2 or 3.

impl<F1, B1, F2, B2, Pad> Convolve<VarDiff<F1, B1>, VarDiff<F2, B2>, Pad> for VarDiff<F1, B1> where
    F1: NData + Debug + Display + 'static,
    F1::Dim: RemoveAxis,
    <F1::Dim as Dimension>::Smaller: RemoveAxis,
    <<F1::Dim as Dimension>::Smaller as Dimension>::Smaller: ReflPad + ReplPad,
    B1: Gradient<Dim = F1::Dim> + Overwrite,
    F2: NData<Dim = F1::Dim> + Debug + Display + 'static,
    B2: Gradient<Dim = F2::Dim> + Overwrite,
    Pad: PaddingMode + 'static, 

type Output = VarDiff<Convolution<F1, F2, Pad>, ConvolutionBackward<F1, B1, F2, B2, Pad>>

The type of the convolution’s result. See the differentiability arithmetic for more details.

fn convolve(
    input: Self,
    kernel: VarDiff<F2, B2>,
    stride: &[usize],
    dilation: &[usize],
    padding: &[usize],
    padding_mode: Pad
) -> Self::Output

Applies a n-dimensional convolution with the given parameters. n can be either 1, 2 or 3.

impl<F1, F2, B2, Pad> ConvolveWithGroups<Var<F1>, VarDiff<F2, B2>, Pad> for Var<F1> where
    F1: NData + 'static,
    F1::Dim: RemoveAxis,
    <F1::Dim as Dimension>::Smaller: RemoveAxis,
    <<F1::Dim as Dimension>::Smaller as Dimension>::Smaller: ReflPad + ReplPad,
    F2: NData<Dim = F1::Dim> + 'static,
    B2: Gradient<Dim = F2::Dim> + Overwrite,
    Pad: PaddingMode + 'static, 

type Output = VarDiff<GroupedConvolution<F1, F2, Pad>, GroupedConvolutionBackwardUnary<F1, B2, Pad>>

The type of the grouped convolution’s result. See the differentiability arithmetic for more details.

fn convolve_with_groups(
    input: Self,
    kernel: VarDiff<F2, B2>,
    stride: &[usize],
    dilation: &[usize],
    padding: &[usize],
    padding_mode: Pad,
    groups: usize
) -> Self::Output

Applies a n-dimensional grouped convolution with the given parameters. n can be either 1, 2 or 3.

impl<F1, B1, F2, B2, Pad> ConvolveWithGroups<VarDiff<F1, B1>, VarDiff<F2, B2>, Pad> for VarDiff<F1, B1> where
    F1: NData + Debug + Display + 'static,
    F1::Dim: RemoveAxis,
    <F1::Dim as Dimension>::Smaller: RemoveAxis,
    <<F1::Dim as Dimension>::Smaller as Dimension>::Smaller: ReflPad + ReplPad,
    B1: Gradient<Dim = F1::Dim> + Overwrite,
    F2: NData<Dim = F1::Dim> + Debug + Display + 'static,
    B2: Gradient<Dim = F2::Dim> + Overwrite,
    Pad: PaddingMode + 'static, 

type Output = VarDiff<GroupedConvolution<F1, F2, Pad>, GroupedConvolutionBackward<F1, B1, F2, B2, Pad>>

The type of the grouped convolution’s result. See the differentiability arithmetic for more details.

fn convolve_with_groups(
    input: Self,
    kernel: VarDiff<F2, B2>,
    stride: &[usize],
    dilation: &[usize],
    padding: &[usize],
    padding_mode: Pad,
    groups: usize
) -> Self::Output

Applies a n-dimensional grouped convolution with the given parameters. n can be either 1, 2 or 3.

impl<T, U> Debug for VarDiff<T, U> where
    T: Data + Debug,
    U: Gradient<Dim = T::Dim> + Overwrite + Debug, 

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: Data + Display, U: Gradient + Overwrite + Display> Display for VarDiff<T, U>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F1, B1, F2> Div<Var<F2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<F2::Dim>, 

type Output = VarDiff<Division<F1, F2>, DivisionBackwardLeft<B1, F2>>

The resulting type after applying the / operator.

fn div(self, rhs: Var<F2>) -> Self::Output

Performs the / operation.

impl<F1, F2, B2> Div<VarDiff<F2, B2>> for Var<F1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    F1::Dim: Dimension + DimMax<B2::Dim>, 

type Output = VarDiff<Division<F1, F2>, DivisionBackwardRight<F1, F2, B2>>

The resulting type after applying the / operator.

fn div(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the / operation.

impl<F1, B1, F2, B2> Div<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<B2::Dim>, 

type Output = VarDiff<Division<F1, F2>, DivisionBackward<F1, B1, F2, B2>>

The resulting type after applying the / operator.

fn div(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the / operation.

impl<T, U> Div<VarDiff<T, U>> for f32 where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    Ix1: DimMax<T::Dim>,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Division<Input<Ix1>, T>, DivisionBackwardRight<Input<Ix1>, T, U>>

The resulting type after applying the / operator.

fn div(self, rhs: VarDiff<T, U>) -> Self::Output

Performs the / operation.

impl<T, U> Div<f32> for VarDiff<T, U> where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Division<T, Input<Ix1>>, DivisionBackwardLeft<U, Input<Ix1>>>

The resulting type after applying the / operator.

fn div(self, rhs: f32) -> Self::Output

Performs the / operation.

impl<T, U> DropoutInput for VarDiff<T, U> where
    T: Data + Forward,
    U: Gradient<Dim = T::Dim> + Overwrite + Backward, 

type Output = VarDiff<DropoutNode<T>, DropoutBackwardNode<U, T>>

fn dropout(self, p: f64, status: Rc<Cell<bool>>) -> Self::Output

impl<F1, B1, F2> MatMatMul<Var<F2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix2> + 'static,
    B1: Gradient<Dim = Ix2> + Overwrite + 'static,
    F2: Data<Dim = Ix2> + 'static, 

type Output = VarDiff<MatrixMatrixMul<F1, F2>, MatrixMatrixMulBackwardLeft<B1, F2>>

The type of the matrix-matrix multiplication’s result. See the differentiability arithmetic for more details.

fn mm(self, rhs: Var<F2>) -> Self::Output

Computes the matrix-matrix multiplication between self and other.

impl<F1, F2, B2> MatMatMul<VarDiff<F2, B2>> for Var<F1> where
    F1: Data<Dim = Ix2> + 'static,
    F2: Data<Dim = Ix2> + 'static,
    B2: Gradient<Dim = Ix2> + Overwrite + 'static, 

type Output = VarDiff<MatrixMatrixMul<F1, F2>, MatrixMatrixMulBackwardRight<F1, B2>>

The type of the matrix-matrix multiplication’s result. See the differentiability arithmetic for more details.

fn mm(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the matrix-matrix multiplication between self and other.

impl<F1, B1, F2, B2> MatMatMul<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix2> + 'static,
    B1: Gradient<Dim = Ix2> + Overwrite + 'static,
    F2: Data<Dim = Ix2> + 'static,
    B2: Gradient<Dim = Ix2> + Overwrite + 'static, 

type Output = VarDiff<MatrixMatrixMul<F1, F2>, MatrixMatrixMulBackward<F1, B1, F2, B2>>

The type of the matrix-matrix multiplication’s result. See the differentiability arithmetic for more details.

fn mm(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the matrix-matrix multiplication between self and other.

impl<F1, B1, F2> MatMatMulT<Var<F2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix2> + 'static,
    B1: Gradient<Dim = Ix2> + Overwrite + 'static,
    F2: Data<Dim = Ix2> + 'static, 

type Output = VarDiff<MatrixMatrixMulT<F1, F2>, MatrixMatrixMulTBackwardLeft<B1, F2>>

The type of the matrix-matrix multiplication with transposed right hand side operand’s result. See the differentiability arithmetic for more details.

fn mm_t(self, rhs: Var<F2>) -> Self::Output

Computes the matrix-matrix multiplication between self and transposed other.

impl<F1, F2, B2> MatMatMulT<VarDiff<F2, B2>> for Var<F1> where
    F1: Data<Dim = Ix2> + 'static,
    F2: Data<Dim = Ix2> + 'static,
    B2: Gradient<Dim = Ix2> + Overwrite + 'static, 

type Output = VarDiff<MatrixMatrixMulT<F1, F2>, MatrixMatrixMulTBackwardRight<F1, B2>>

The type of the matrix-matrix multiplication with transposed right hand side operand’s result. See the differentiability arithmetic for more details.

fn mm_t(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the matrix-matrix multiplication between self and transposed other.

impl<F1, B1, F2, B2> MatMatMulT<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix2> + 'static,
    B1: Gradient<Dim = Ix2> + Overwrite + 'static,
    F2: Data<Dim = Ix2> + 'static,
    B2: Gradient<Dim = Ix2> + Overwrite + 'static, 

type Output = VarDiff<MatrixMatrixMulT<F1, F2>, MatrixMatrixMulTBackward<F1, B1, F2, B2>>

The type of the matrix-matrix multiplication with transposed right hand side operand’s result. See the differentiability arithmetic for more details.

fn mm_t(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the matrix-matrix multiplication between self and transposed other.

impl<F1, B1, F2> Mul<Var<F2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<F2::Dim>, 

type Output = VarDiff<Multiplication<F1, F2>, MultiplicationBackwardUnary<B1, F2>>

The resulting type after applying the * operator.

fn mul(self, rhs: Var<F2>) -> Self::Output

Performs the * operation.

impl<F1, F2, B2> Mul<VarDiff<F2, B2>> for Var<F1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B2::Dim: Dimension + DimMax<F1::Dim>, 

type Output = VarDiff<Multiplication<F1, F2>, MultiplicationBackwardUnary<B2, F1>>

The resulting type after applying the * operator.

fn mul(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the * operation.

impl<F1, B1, F2, B2> Mul<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<B2::Dim>, 

type Output = VarDiff<Multiplication<F1, F2>, MultiplicationBackward<F1, B1, F2, B2>>

The resulting type after applying the * operator.

fn mul(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the * operation.

impl<T, U> Mul<VarDiff<T, U>> for f32 where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    Ix1: DimMax<T::Dim>,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Multiplication<Input<Ix1>, T>, MultiplicationBackwardUnary<U, Input<Ix1>>>

The resulting type after applying the * operator.

fn mul(self, rhs: VarDiff<T, U>) -> Self::Output

Performs the * operation.

impl<T, U> Mul<f32> for VarDiff<T, U> where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Multiplication<T, Input<Ix1>>, MultiplicationBackwardUnary<U, Input<Ix1>>>

The resulting type after applying the * operator.

fn mul(self, rhs: f32) -> Self::Output

Performs the * operation.

impl<T, U> Neg for VarDiff<T, U> where
    T: Data + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static, 

type Output = VarDiff<Negation<T>, NegationBackward<U>>

The resulting type after applying the - operator.

fn neg(self) -> Self::Output

Performs the unary - operation.

impl<T, U> Register for VarDiff<T, U> where
    T: Data + 'static,
    U: Gradient + Overwrite + 'static, 

fn register_params(&self, params: &mut Vec<RawParam>)

Registers self’s parameters to the model’s status parameters params.

fn register_status(&mut self, _: Rc<Cell<bool>>)

Register self’s status to the model’s status state status.

impl<F1, B1, F2> Stack<Var<F2>> for VarDiff<F1, B1> where
    F1: Data<Dim = B1::Dim> + 'static,
    F2: Data<Dim = F1::Dim> + 'static,
    B1: Gradient + Overwrite + 'static,
    F1::Dim: RemoveAxis,
    B1::Dim: RemoveAxis, 

type Output = VarDiff<Stack<F1, F2>, StackBackwardLeft<B1>>

The type of the stacking’s result. See the differentiability arithmetic for more details.

fn stack(self, rhs: Var<F2>, axis: usize) -> Self::Output

Stacks variables along the given axis.

impl<F1, F2, B2> Stack<VarDiff<F2, B2>> for Var<F1> where
    F1: Data + 'static,
    F2: Data<Dim = F1::Dim> + 'static,
    B2: Gradient<Dim = F1::Dim> + Overwrite + 'static,
    B2::Dim: RemoveAxis,
    F1::Dim: RemoveAxis, 

type Output = VarDiff<Stack<F1, F2>, StackBackwardRight<B2>>

The type of the stacking’s result. See the differentiability arithmetic for more details.

fn stack(self, rhs: VarDiff<F2, B2>, axis: usize) -> Self::Output

Stacks variables along the given axis.

impl<F1, B1, F2, B2> Stack<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    F2: Data<Dim = F1::Dim> + 'static,
    B2: Gradient<Dim = B1::Dim> + Overwrite + 'static,
    F1::Dim: RemoveAxis,
    B1::Dim: RemoveAxis, 

type Output = VarDiff<Stack<F1, F2>, StackBackward<B1, B2>>

The type of the stacking’s result. See the differentiability arithmetic for more details.

fn stack(self, rhs: VarDiff<F2, B2>, axis: usize) -> Self::Output

Stacks variables along the given axis.

impl<F1, B1, F2> Sub<Var<F2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<F2::Dim>, 

type Output = VarDiff<Subtraction<F1, F2>, SubtractionBackwardLeft<B1, F2>>

The resulting type after applying the - operator.

fn sub(self, rhs: Var<F2>) -> Self::Output

Performs the - operation.

impl<F1, F2, B2> Sub<VarDiff<F2, B2>> for Var<F1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B2::Dim: Dimension + DimMax<F1::Dim>, 

type Output = VarDiff<Subtraction<F1, F2>, SubtractionBackwardRight<B2, F1>>

The resulting type after applying the - operator.

fn sub(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the - operation.

impl<F1, B1, F2, B2> Sub<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data + 'static,
    F2: Data + 'static,
    B1: Gradient + Overwrite + 'static,
    B2: Gradient + Overwrite + 'static,
    F1::Dim: Dimension + DimMax<F2::Dim>,
    B1::Dim: Dimension + DimMax<B2::Dim>, 

type Output = VarDiff<Subtraction<F1, F2>, SubtractionBackward<B1, B2>>

The resulting type after applying the - operator.

fn sub(self, rhs: VarDiff<F2, B2>) -> Self::Output

Performs the - operation.

impl<T, U> Sub<VarDiff<T, U>> for f32 where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    Ix1: DimMax<T::Dim>,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Subtraction<Input<Ix1>, T>, SubtractionBackwardRight<U, Input<Ix1>>>

The resulting type after applying the - operator.

fn sub(self, rhs: VarDiff<T, U>) -> Self::Output

Performs the - operation.

impl<T, U> Sub<f32> for VarDiff<T, U> where
    T: Data + Forward + 'static,
    U: Gradient<Dim = T::Dim> + Overwrite + 'static,
    T::Dim: DimMax<Ix1>, 

type Output = VarDiff<Subtraction<T, Input<Ix1>>, SubtractionBackwardLeft<U, Input<Ix1>>>

The resulting type after applying the - operator.

fn sub(self, rhs: f32) -> Self::Output

Performs the - operation.

impl<F1, B1, F2> VecMatMul<Var<F2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix1> + 'static,
    B1: Gradient<Dim = Ix1> + Overwrite + 'static,
    F2: Data<Dim = Ix2> + 'static, 

type Output = VarDiff<VectorMatrixMul<F1, F2>, VectorMatrixMulBackwardLeft<B1, F2>>

The type of the vector-matrix multiplication’s result. See the differentiability arithmetic for more details.

fn vm(self, rhs: Var<F2>) -> Self::Output

Computes the vector-matrix multiplication between self and other.

impl<F1, F2, B2> VecMatMul<VarDiff<F2, B2>> for Var<F1> where
    F1: Data<Dim = Ix1> + 'static,
    F2: Data<Dim = Ix2> + 'static,
    B2: Gradient<Dim = Ix2> + Overwrite + 'static, 

type Output = VarDiff<VectorMatrixMul<F1, F2>, VectorMatrixMulBackwardRight<F1, B2>>

The type of the vector-matrix multiplication’s result. See the differentiability arithmetic for more details.

fn vm(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the vector-matrix multiplication between self and other.

impl<F1, B1, F2, B2> VecMatMul<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix1> + 'static,
    B1: Gradient<Dim = Ix1> + Overwrite + 'static,
    F2: Data<Dim = Ix2> + 'static,
    B2: Gradient<Dim = Ix2> + Overwrite + 'static, 

type Output = VarDiff<VectorMatrixMul<F1, F2>, VectorMatrixMulBackward<F1, B1, F2, B2>>

The type of the vector-matrix multiplication’s result. See the differentiability arithmetic for more details.

fn vm(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the vector-matrix multiplication between self and other.

impl<F1, B1, F2> VecVecMul<Var<F2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix1> + 'static,
    B1: Gradient<Dim = Ix1> + Overwrite + 'static,
    F2: Data<Dim = Ix1> + 'static, 

type Output = VarDiff<VectorVectorMul<F1, F2>, VectorVectorMulBackwardUnary<B1, F2>>

The type of the dot product’s result. See the differentiability arithmetic for more details.

fn vv(self, rhs: Var<F2>) -> Self::Output

Computes the dot product between self and other.

impl<F1, F2, B2> VecVecMul<VarDiff<F2, B2>> for Var<F1> where
    F1: Data<Dim = Ix1> + 'static,
    F2: Data<Dim = Ix1> + 'static,
    B2: Gradient<Dim = Ix1> + Overwrite + 'static, 

type Output = VarDiff<VectorVectorMul<F1, F2>, VectorVectorMulBackwardUnary<B2, F1>>

The type of the dot product’s result. See the differentiability arithmetic for more details.

fn vv(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the dot product between self and other.

impl<F1, B1, F2, B2> VecVecMul<VarDiff<F2, B2>> for VarDiff<F1, B1> where
    F1: Data<Dim = Ix1> + 'static,
    B1: Gradient<Dim = Ix1> + Overwrite + 'static,
    F2: Data<Dim = Ix1> + 'static,
    B2: Gradient<Dim = Ix1> + Overwrite + 'static, 

type Output = VarDiff<VectorVectorMul<F1, F2>, VectorVectorMulBackward<F1, B1, F2, B2>>

The type of the dot product’s result. See the differentiability arithmetic for more details.

fn vv(self, rhs: VarDiff<F2, B2>) -> Self::Output

Computes the dot product between self and other.