[−][src]Trait mli::Backward

pub trait Backward: Forward {
    type OutputDelta;
    type InputDelta;
    type TrainDelta;
    fn backward(
        &self, 
        input: &Self::Input, 
        internal: &Self::Internal, 
        output_delta: &Self::OutputDelta
    ) -> (Self::InputDelta, Self::TrainDelta);

    fn backward_input(
        &self, 
        input: &Self::Input, 
        internal: &Self::Internal, 
        output_delta: &Self::OutputDelta
    ) -> Self::InputDelta { ... }

    fn backward_train(
        &self, 
        input: &Self::Input, 
        internal: &Self::Internal, 
        output_delta: &Self::OutputDelta
    ) -> Self::TrainDelta { ... }
}

This trait indicates support of backwards propogation.

This trait also contains methods to perform training if training is possible. If training is not possible, this trait can still be implemented with those definitions being empty. In that case, machine learning algorithms will still be able to back propogate over this operation, but training it will be a no-op.

Associated Types

type OutputDelta

type InputDelta

type TrainDelta

Loading content...

Required methods

fn backward(
    &self,
    input: &Self::Input,
    internal: &Self::Internal,
    output_delta: &Self::OutputDelta
) -> (Self::InputDelta, Self::TrainDelta)

partials produces the change required in the input and trainable variables.

Key:

  • f is the output
  • x is the input
  • v is the trainable variables
  • E is the loss or error
  • delta is equivalent to Δf or -η * 𝛿E/𝛿f

This method should produce (Δx, Δv):

  • Δx = Δf * 𝛿f/𝛿x
  • Δv = Δf * 𝛿f/𝛿v

𝛿f/𝛿x and 𝛿f/𝛿v can be an approximation, particularly if the function is not differentiable (e.g. ReLU and signum).

Loading content...

Provided methods

fn backward_input(
    &self,
    input: &Self::Input,
    internal: &Self::Internal,
    output_delta: &Self::OutputDelta
) -> Self::InputDelta

See Backward::backward for documentation.

fn backward_train(
    &self,
    input: &Self::Input,
    internal: &Self::Internal,
    output_delta: &Self::OutputDelta
) -> Self::TrainDelta

See Backward::backward for documentation.

Loading content...

Implementors

impl<'a, T> Backward for &'a T where
    T: Backward, [src]

type OutputDelta = T::OutputDelta

type InputDelta = T::InputDelta

type TrainDelta = T::TrainDelta

impl<'a, T> Backward for &'a mut T where
    T: Backward, [src]

type OutputDelta = T::OutputDelta

type InputDelta = T::InputDelta

type TrainDelta = T::TrainDelta

impl<T, U, O> Backward for Chain<T, U> where
    T: Backward<OutputDelta = U::InputDelta> + Forward<Output = O>,
    U: Backward + Forward<Input = O>, [src]

type OutputDelta = U::OutputDelta

type InputDelta = T::InputDelta

type TrainDelta = (T::TrainDelta, U::TrainDelta)

Loading content...