Crate concision_core

Expand description

This crate provides the core implementations for the cnc framework, defining various traits, types, and utilities essential for building neural networks.

ParamsBase: A structure for defining the parameters within a neural network.
Backward: This trait establishes a common interface for backward propagation.
Forward: This trait denotes a single forward pass through a layer of a neural network

§Features

The crate is heavily feature-gate, enabling users to customize their experience based on their needs.

init: Enables (random) initialization routines for models, parameters, and tensors.
utils: Provides various utilities for developing machine learning models.

§Dependency-specific Features

Additionally, the crate provides various dependency-specific features that can be enabled:

anyhow: Enables the use of the anyhow crate for error handling.
approx: Enables approximate equality checks for floating-point numbers.
complex: Enables complex number support.
json: Enables JSON serialization and deserialization capabilities.
rand: Enables random number generation capabilities.
serde: Enables serialization and deserialization capabilities.
tracing: Enables tracing capabilities for debugging and logging.

Re-exports§

pub use rand;
pub use rand_distr;
pub use super::error::ParamsError;
pub use ndtensor as tensor;

Modules§

activate: this module is dedicated to activation function Activation functions for neural networks and their components. These functions are often used to introduce non-linearity into the model, allowing it to learn more complex patterns in the data.
error: this module provides the base Error type for the library This module implements the core Error type for the framework and provides a Result type alias for convenience.
init: this module establishes generic random initialization routines for models, params, and tensors.
loss: this module focuses on the loss functions used in training neural networks.
ops: This module provides the core operations for tensors, including filling, padding, reshaping, and tensor manipulation.
params: this module provides the ParamsBase type for the library, which is used to define the parameters of a neural network. Parameters for constructing neural network models. This module implements parameters using the ParamsBase struct and its associated types. The ParamsBase struct provides:
traits: This module provides the core traits for the library, such as Backward and Forward
utils: this module implements various utilities useful for developing machine learning models A suite of utilities tailored toward neural networks.

Structs§

LecunNormal: LecunNormal is a truncated normal distribution centered at 0 with a standard deviation that is calculated as:
PadActionIter: An iterator over the variants of PadAction
Padding
ParamsBase: The ParamsBase struct is a generic container for a set of weights and biases for a model where the bias tensor is always n-1 dimensions smaller than the weights tensor. Consequently, this constrains the ParamsBase implementation to only support dimensions that can be reduced by one axis (i.e. $\mbox{rank}(D)>0$), which is typically the “zero-th” axis.
TensorBase: the TensorBase struct is the base type for all tensors in the library.
TruncatedNormal: The TruncatedNormal distribution is similar to the StandardNormal distribution, differing in that is computes a boundary equal to two standard deviations from the mean. More formally, the boundary is defined as:
XavierNormal: Normal Xavier initializers leverage a normal distribution centered around 0 and using a standard deviation ($\sigma$) computed by:
XavierUniform: Uniform Xavier initializers use a uniform distribution to initialize the weights of a neural network within a given range.

Enums§

Error: The Error type enumerates various errors that can occur within the framework.
PadAction
PadError
PadMode

Traits§

Affine: apply an affine transformation to a tensor; affine transformation is defined as mul * self + add
Apply: The Apply establishes an interface for owned containers that are capable of applying some function onto their elements.
ApplyGradient: A trait declaring basic gradient-related routines for a neural network
ApplyGradientExt: This trait extends the ApplyGradient trait by allowing for momentum-based optimization
ApplyMut: The ApplyMut trait mutates the each element of the container, in-place, using the given function.
ArrayLike
AsBiasDim: The AsBiasDim trait is used to define a type that can be used to get the bias dimension of the parameters.
AsComplex
Backward: Backward propagate a delta through the system;
Biased
CallInPlace: The [CallOnMut] is a supertrait of the CallInto trait that enables an object to be passed onto a unary, or single value, function that is applied to the object, but with the ability to mutate the object in-place.
CallInto: The CallInto trait is a consuming interface for passing an object into a single-valued function. While the intended affect is the same as CallOn, the difference is that CallInto enables a transfer of ownership instead of relyin upon a reference.
CallOn: The CallOn trait enables an object to be passed onto a unary, or single value, function that is applied to the object.
Clip: A trait denoting objects capable of being clipped between some minimum and some maximum.
ClipMut: This trait enables tensor clipping; it is implemented for ArrayBase
Codex
CrossEntropy: A trait for computing the cross-entropy loss of a tensor or array
Decode: Decode defines a standard interface for decoding data.
DecrementAxis: The DecrementAxis trait defines a method enabling an axis to decrement itself,
DefaultLike
DropOut: [Dropout] randomly zeroizes elements with a given probability (p).
Encode: Encode defines a standard interface for encoding data.
FillLike
FloorDiv
Forward: This trait denotes entities capable of performing a single forward step
Gradient: the Gradient trait defines the gradient of a function, which is a function that takes an input and returns a delta, which is the change in the output with respect to the input.
Heavyside
IncrementAxis: The IncrementAxis trait defines a method enabling an axis to increment itself, effectively adding a new axis to the array.
Init: A trait for creating custom initialization routines for models or other entities.
InitInplace: This trait enables models to implement custom, in-place initialization methods.
Initialize: This trait provides the base methods required for initializing tensors with random values. The trait is similar to the RandomExt trait provided by the ndarray_rand crate, however, it is designed to be more generic, extensible, and optimized for neural network initialization routines. Initialize is implemented for ArrayBase as well as ParamsBase allowing you to randomly initialize new tensors and parameters.
IntoAxis: The IntoAxis trait is used to define a conversion routine that takes a type and wraps it in an Axis type.
IntoComplex: Trait for converting a type into a complex number.
Inverse: The Inverse trait generically establishes an interface for computing the inverse of a type, regardless of if its a tensor, scalar, or some other compatible type.
IsSquare: IsSquare is a trait for checking if the layout, or dimensionality, of a tensor is square.
L1Norm: a trait for computing the L1 norm of a tensor or array
L2Norm: a trait for computing the L2 norm of a tensor or array
LinearActivation
Loss: The Loss trait defines a common interface for any custom loss function implementations. This trait requires the implementor to define their algorithm for calculating the loss between two values, lhs and rhs, which can be of different types, X and Y respectively. These terms are used generically to allow for flexibility in the allowed types, such as tensors, scalars, or other data structures while clearly defining the “order” in which the operations are performed. It is most common to expect the lhs to be the predicted output and the rhs to be the actual output, but this is not a strict requirement. The trait also defines an associated type Output, which represents the type of the loss value returned by the loss method. This allows for different loss functions to return different types of loss values, such as scalars or tensors, depending on the specific implementation of the loss function.
MaskFill: This trait is used to fill an array with a value based on a mask. The mask is a boolean array of the same shape as the array.
MatMul: The MatMul trait defines an interface for matrix multiplication.
MatPow: The MatPow trait defines an interface for computing the exponentiation of a matrix.
MeanAbsoluteError: Compute the mean absolute error (MAE) of the object; more formally, we define the MAE as the average of the absolute differences between the predicted and actual values:
MeanSquaredError: The MeanSquaredError (MSE) is the average of the squared differences between the ($$\hat{y_{i}}$$) and actual values ($y_{i}$):
NdLike
NdTensor: The [Tensor] trait extends the RawTensor trait to provide additional functionality for tensors, such as creating tensors from shapes, applying functions, and iterating over elements. It is generic over the element type A and the dimension type `D
Norm: The Norm trait serves as a unified interface for various normalization routnines. At the moment, the trait provides L1 and L2 techniques.
Numerical: Numerical is a trait for all numerical types; implements a number of core operations
OnesLike
Pad: The Pad trait defines a padding operation for tensors.
PercentDiff: Compute the percentage difference between two values. The percentage difference is defined as:
RawDimension: the RawDimension trait is used to define a type that can be used as a raw dimension. This trait is primarily used to provide abstracted, generic interpretations of the dimensions of the ndarray crate to ensure long-term compatibility.
RawStore: The RawStore trait provides a generalized interface for all containers. The trait is sealed, preventing any external implementations and is primarily used as the basis for other traits, such as Sequential.
RawTensor: The RawTensor trait defines the base interface for all tensors,
ReLU
Rho: The Rho trait defines a set of activation functions that can be applied to an implementor of the Apply trait. It provides methods for common activation functions such as linear, heavyside, ReLU, sigmoid, and tanh, along with their derivatives. The trait is generic over a type U, which represents the data type of the input to the activation functions. The trait also inherits a type alias Cont<U> to allow for variance w.r.t. the outputs of defined methods.
RhoComplex: The RhoComplex trait is similar to the Rho trait in that it provides various activation functions for implementos of the Apply trait, however, instead of being truly generic over a type U, it is generic over a type U that implements the ComplexFloat trait. This enables the use of complex numbers in the activation functions, something particularly useful for signal-based workloads.
Root: The Root trait provides methods for computing the nth root of a number.
RoundTo
Scalar: The Scalar trait extends the Numerical trait to include additional mathematical operations for the purpose of reducing the number of overall traits required to complete various machine-learning tasks.
ScalarComplex
Sequential: The Sequential trait is a marker trait defining a sequential collection of elements. It is sealed, preventing external implementations, and is used to indicate that a type can be treated as a sequence of elements, such as arrays or vectors.
Sigmoid
Softmax
SoftmaxAxis
SummaryStatistics: This trait describes the fundamental methods of summary statistics. These include the mean, standard deviation, variance, and more.
Tanh
Transpose: The Transpose trait generically establishes an interface for transposing a type
Unsqueeze: The Unsqueeze trait establishes an interface for a routine that unsqueezes an array, by inserting a new axis at a specified position. This is useful for reshaping arrays to meet specific dimensional requirements.
Weighted
ZerosLike

Functions§

calculate_pattern_similarity: Calculate similarity between two patterns
clip_gradient: Clip the gradient to a maximum value.
clip_inf_nan
concat_iter: Creates an n-dimensional array from an iterator of n dimensional arrays.
extract_patterns: Extract common patterns from historical sequences
floor_div: divide two values and round down to the nearest integer.
genspace
heavyside: Heaviside activation function:
hstack: stack a 1D array into a 2D array by stacking them horizontally.
inverse
is_similar_pattern: Check if two patterns are similar enough to be considered duplicates
layer_norm
layer_norm_axis
linarr
linear: the linear method is essentially a passthrough method often used in simple models or layers where no activation is needed.
linear_derivative: the linear_derivative method always returns 1 as it is a simple, single variable function
pad
pad_to
randc: Generate a random array of complex numbers with real and imaginary parts in the range [0, 1)
relu: the relu activation function:
relu_derivative
round_to: Round the given value to the given number of decimal places.
sigmoid: the sigmoid activation function:
sigmoid_derivative: the derivative of the sigmoid function
softmax: Softmax function:
softmax_axis: Softmax function along a specific axis:
stack_iter: Creates a larger array from an iterator of smaller arrays.
stdnorm: Given a shape, generate a random array using the StandardNormal distribution
stdnorm_from_seed
tanh: the tanh activation function:
tanh_derivative: the derivative of the tanh function
tril: Returns the lower triangular portion of a matrix.
triu: Returns the upper triangular portion of a matrix.
uniform_from_seed: Creates a random array from a uniform distribution using a given key
vstack: stack a 1D array into a 2D array by stacking them vertically.

Type Aliases§

ArcParams: A type alias for shared parameters
ArcTensor: a type alias for a TensorBase setup to use a shared, thread-safe internal representation of the data.
CowParams: A type alias for a ParamsBase with a borrowed internal layout
CowTensor: a type alias for a TensorBase using a borrowed layout
PadResult
Params: A type alias for a ParamsBase with an owned internal layout
ParamsView: A type alias for an immutable view of the parameters
ParamsViewMut: A type alias for a mutable view of the parameters
RawMutParams: A type alias for the ParamsBase whose elements are of type *mut A using a RawViewRepr layout
RawViewMutTensor: a type alias for a TensorBase with an owned representation
RawViewParams: A type alias for the ParamsBase whose elements are of type *const A using a RawViewRepr layout
RawViewTensor: A type alias for a TensorBase with a raw pointer representation.
Result: a type alias for a Result configured with an Error as its error type.
Tensor: a type alias for a TensorBase with an owned representation
TensorView: a type alias for a TensorBase with a view representation
TensorViewMut: a type alias for a TensorBase with a mutable view representation