Module multistochgrad::types

This file is inspired by the crate optimisation written by Oliver Mader b52@reaktor42.de.
I kept the traits Function, FunctionC1, Summation and SummationC1 which provides the interface for users to define a minimisation problem.

In fact when minimising summation function we often seek to minimize the mean of the terms which is the same but scales gradient and this makes the implementation of batched stochastic gradient more natural as we always computes mean gradient over terms taken into account.

I kept the names of methods and reimplemented according to the following changes:

1. In batched stochastic gradient we need to define mean gradient on a subset of indexes.
2. I use the crate ndarray which provides addition of vector and enables rayon for //
3. I got rid of the iterator on indexes as indexes are always usize.
4. The function minimize in Trait Minimizer takes a generic Argument for future extension
5. I use rayon to compute value of summation for values and gradients with parallel iterators

Structs

• Solution
  A solution of a minimization run providing only the minimal information.

Traits

• Evaluation
  Captures the essence of a function evaluation.
• Function
  Defines an objective function f that is subject to minimization. trait must be sync to be able to be sent over threads
• FunctionC1
  Defines an objective function f that is able to compute the first derivative f'(x).
• Minimizer
  Defines an optimizer that is able to minimize a given objective function F.
• Summation
  Defines a summation of individual functions, i.e., f(x) = ∑ᵢ fᵢ(x).
• SummationC1
  Defines a summation of individual functions fᵢ(x), assuming that each function has a first derivative.

Functions

• norm_l2