Expand description
§concision (cnc)
concision
aims to be a complete machine-learning toolkit written in Rust. The framework
is designed to be performant, extensible, and easy to use while offering a wide range of
features for building and training machine learning models.
The framework relies heavily on the ndarray
crate for its
n-dimensional arrays, which are essential for efficient data manipulation and mathematical
operations.
§Features
data
: Provides utilities for data loading, preprocessing, and augmentation.derive
: Custom derive macros for automatic implementation of traitsinit
: Enables various initialization strategies for model parameters.macros
: Procedural macros for simplifying common tasks in machine learning.neural
: A neural network module that includes layers, optimizers, and training utilities.
§Extensions
The crate is integrated with several optional externcal crates that are commonly used in Rust development; listed below are some of the most relevant of these extensions as they add additional functionality to the framework.
approx
: Enables approximate equality checks for floating-point arithmetic, useful for testing and validation of model outputs.json
: Enables JSON serialization and deserialization for models and data.rayon
: Enables parallel processing for data loading and training.serde
: Enables theserde
crate for the serialization and deserialization of models and data.tracing
: Enables thetracing
crate for structured logging and diagnostics.
§Roadmap
- DSL: Create a pseudo-DSL for defining machine learning models and training processes.
- GPU: Support for GPU acceleration to speed up training and inference.
- Interoperability: Integrate with other libraries and frameworks (TensorFlow, PyTorch)
- Visualization: Utilities for visualizing model architectures and training progress
- WASM: Native support for WebAssembly enabling models to be run in web browsers.
Modules§
- activate
- this module is dedicated to activation function Activation functions for neural networks and their components. These functions are often used to introduce non-linearity into the model, allowing it to learn more complex patterns in the data.
- data
- this module contains various data loaders, preprocessors, and augmenters Datasets and data loaders for the Concision framework.
- error
- this module provides the base
Error
type for the library This module implements the coreError
type for the framework and provides aResult
type alias for convenience. - loss
- this module focuses on the loss functions used in training neural networks.
- nn
- this module defines various neural network abstractions, layers, and training utilities
Various components, implementations, and traits for creating neural networks. The crate
builds off of the
concision_core
crate, making extensive use of theParamsBase
type to define the parameters of layers within a network. - ops
- This module provides the core operations for tensors, including filling, padding, reshaping, and tensor manipulation.
- params
- this module provides the
ParamsBase
type for the library, which is used to define the parameters of a neural network. Parameters for constructing neural network models. This module implements parameters using the ParamsBase struct and its associated types. The ParamsBase struct provides: - rand
- Utilities for random number generation
- rand_
distr - Generating random samples from probability distributions.
- tensor
- ndtensor
- traits
- This module provides the core traits for the library, such as
Backward
andForward
- utils
- A suite of utilities tailored toward neural networks.
Macros§
- model
- the
model!
procedural macro is used to streamline the creation of custom models using theconcision
framework - model_
config model_config!
is a procedural macro used to define the configuration for a model in theconcision
framework. It allows users to specify various parameters and settings for the model in a concise and structured manner, declaring a name for their instanc
Structs§
- PadAction
Iter - An iterator over the variants of PadAction
- Padding
- Params
Base - The
ParamsBase
struct is a generic container for a set of weights and biases for a model where the bias tensor is alwaysn-1
dimensions smaller than theweights
tensor. Consequently, this constrains theParamsBase
implementation to only support dimensions that can be reduced by one axis (i.e. $\mbox{rank}(D)>0
$), which is typically the “zero-th” axis. - Tensor
Base - the
TensorBase
struct is the base type for all tensors in the library.
Enums§
- Error
- The
Error
type enumerates various errors that can occur within the framework. - PadAction
- PadError
- PadMode
- Params
Error - the
ParamsError
enumerates various errors that can occur within the parameters of a neural network.
Traits§
- Affine
- apply an affine transformation to a tensor;
affine transformation is defined as
mul * self + add
- Apply
- The
Apply
establishes an interface for owned containers that are capable of applying some function onto their elements. - Apply
Gradient - A trait declaring basic gradient-related routines for a neural network
- Apply
Gradient Ext - This trait extends the ApplyGradient trait by allowing for momentum-based optimization
- Apply
Mut - The
ApplyMut
trait mutates the each element of the container, in-place, using the given function. - Array
Like - AsBias
Dim - The
AsBiasDim
trait is used to define a type that can be used to get the bias dimension of the parameters. - Backward
- Backward propagate a delta through the system;
- Biased
- Call
InPlace - The [
CallOnMut
] is a supertrait of theCallInto
trait that enables an object to be passed onto a unary, or single value, function that is applied to the object, but with the ability to mutate the object in-place. - Call
Into - The
CallInto
trait is a consuming interface for passing an object into a single-valued function. While the intended affect is the same asCallOn
, the difference is thatCallInto
enables a transfer of ownership instead of relyin upon a reference. - CallOn
- The
CallOn
trait enables an object to be passed onto a unary, or single value, function that is applied to the object. - Clip
- A trait denoting objects capable of being clipped between some minimum and some maximum.
- ClipMut
- This trait enables tensor clipping; it is implemented for
ArrayBase
- Codex
- Cross
Entropy - A trait for computing the cross-entropy loss of a tensor or array
- Decode
- Decode defines a standard interface for decoding data.
- Decrement
Axis - The
DecrementAxis
trait defines a method enabling an axis to decrement itself, - Default
Like - DropOut
- [Dropout] randomly zeroizes elements with a given probability (
p
). - Encode
- Encode defines a standard interface for encoding data.
- Fill
Like - Floor
Div - Forward
- This trait denotes entities capable of performing a single forward step
- Gradient
- the
Gradient
trait defines the gradient of a function, which is a function that takes an input and returns a delta, which is the change in the output with respect to the input. - Heavyside
- Increment
Axis - The
IncrementAxis
trait defines a method enabling an axis to increment itself, effectively adding a new axis to the array. - Into
Axis - The
IntoAxis
trait is used to define a conversion routine that takes a type and wraps it in anAxis
type. - Inverse
- The
Inverse
trait generically establishes an interface for computing the inverse of a type, regardless of if its a tensor, scalar, or some other compatible type. - IsSquare
IsSquare
is a trait for checking if the layout, or dimensionality, of a tensor is square.- L1Norm
- a trait for computing the L1 norm of a tensor or array
- L2Norm
- a trait for computing the L2 norm of a tensor or array
- Linear
Activation - Loss
- The
Loss
trait defines a common interface for any custom loss function implementations. This trait requires the implementor to define their algorithm for calculating the loss between two values,lhs
andrhs
, which can be of different types,X
andY
respectively. These terms are used generically to allow for flexibility in the allowed types, such as tensors, scalars, or other data structures while clearly defining the “order” in which the operations are performed. It is most common to expect thelhs
to be the predicted output and therhs
to be the actual output, but this is not a strict requirement. The trait also defines an associated typeOutput
, which represents the type of the loss value returned by theloss
method. This allows for different loss functions to return different types of loss values, such as scalars or tensors, depending on the specific implementation of the loss function. - Mask
Fill - This trait is used to fill an array with a value based on a mask. The mask is a boolean array of the same shape as the array.
- MatMul
- The
MatMul
trait defines an interface for matrix multiplication. - MatPow
- The
MatPow
trait defines an interface for computing the exponentiation of a matrix. - Mean
Absolute Error - Compute the mean absolute error (MAE) of the object; more formally, we define the MAE as the average of the absolute differences between the predicted and actual values:
- Mean
Squared Error - The
MeanSquaredError
(MSE) is the average of the squared differences between the ($$\hat{y_{i}}$$) and actual values ($y_{i}
$): - NdLike
- NdTensor
- The [
Tensor
] trait extends theRawTensor
trait to provide additional functionality for tensors, such as creating tensors from shapes, applying functions, and iterating over elements. It is generic over the element typeA
and the dimension type `D - Norm
- The Norm trait serves as a unified interface for various normalization routnines. At the moment, the trait provides L1 and L2 techniques.
- Numerical
- Numerical is a trait for all numerical types; implements a number of core operations
- Ones
Like - Pad
- The
Pad
trait defines a padding operation for tensors. - Percent
Diff - Compute the percentage difference between two values. The percentage difference is defined as:
- RawDimension
- the
RawDimension
trait is used to define a type that can be used as a raw dimension. This trait is primarily used to provide abstracted, generic interpretations of the dimensions of thendarray
crate to ensure long-term compatibility. - RawStore
- The
RawStore
trait provides a generalized interface for all containers. The trait is sealed, preventing any external implementations and is primarily used as the basis for other traits, such asSequential
. - RawTensor
- The
RawTensor
trait defines the base interface for all tensors, - ReLU
- Rho
- The
Rho
trait defines a set of activation functions that can be applied to an implementor of theApply
trait. It provides methods for common activation functions such as linear, heavyside, ReLU, sigmoid, and tanh, along with their derivatives. The trait is generic over a typeU
, which represents the data type of the input to the activation functions. The trait also inherits a type aliasCont<U>
to allow for variance w.r.t. the outputs of defined methods. - Root
- The
Root
trait provides methods for computing the nth root of a number. - RoundTo
- Scalar
- The Scalar trait extends the Numerical trait to include additional mathematical operations for the purpose of reducing the number of overall traits required to complete various machine-learning tasks.
- Sequential
- The
Sequential
trait is a marker trait defining a sequential collection of elements. It is sealed, preventing external implementations, and is used to indicate that a type can be treated as a sequence of elements, such as arrays or vectors. - Sigmoid
- Softmax
- Softmax
Axis - Summary
Statistics - This trait describes the fundamental methods of summary statistics. These include the mean, standard deviation, variance, and more.
- Tanh
- Transpose
- The
Transpose
trait generically establishes an interface for transposing a type - Unsqueeze
- The
Unsqueeze
trait establishes an interface for a routine that unsqueezes an array, by inserting a new axis at a specified position. This is useful for reshaping arrays to meet specific dimensional requirements. - Weighted
- Zeros
Like
Functions§
- calculate_
pattern_ similarity - Calculate similarity between two patterns
- clip_
gradient - Clip the gradient to a maximum value.
- clip_
inf_ nan - concat_
iter - Creates an n-dimensional array from an iterator of n dimensional arrays.
- extract_
patterns - Extract common patterns from historical sequences
- floor_
div - divide two values and round down to the nearest integer.
- genspace
- heavyside
- Heaviside activation function:
- hstack
- stack a 1D array into a 2D array by stacking them horizontally.
- inverse
- is_
similar_ pattern - Check if two patterns are similar enough to be considered duplicates
- layer_
norm - layer_
norm_ axis - linarr
- linear
- the
linear
method is essentially a passthrough method often used in simple models or layers where no activation is needed. - linear_
derivative - the
linear_derivative
method always returns1
as it is a simple, single variable function - pad
- pad_to
- relu
- the relu activation function:
- relu_
derivative - round_
to - Round the given value to the given number of decimal places.
- sigmoid
- the sigmoid activation function:
- sigmoid_
derivative - the derivative of the sigmoid function
- softmax
- Softmax function:
- softmax_
axis - Softmax function along a specific axis:
- stack_
iter - Creates a larger array from an iterator of smaller arrays.
- tanh
- the tanh activation function:
- tanh_
derivative - the derivative of the tanh function
- tril
- Returns the lower triangular portion of a matrix.
- triu
- Returns the upper triangular portion of a matrix.
- vstack
- stack a 1D array into a 2D array by stacking them vertically.
Type Aliases§
- ArcParams
- A type alias for shared parameters
- ArcTensor
- a type alias for a
TensorBase
setup to use a shared, thread-safe internal representation of the data. - CowParams
- A type alias for a
ParamsBase
with a borrowed internal layout - CowTensor
- a type alias for a
TensorBase
using a borrowed layout - PadResult
- Params
- A type alias for a
ParamsBase
with an owned internal layout - Params
View - A type alias for an immutable view of the parameters
- Params
View Mut - A type alias for a mutable view of the parameters
- RawMut
Params - A type alias for the
ParamsBase
whose elements are of type*mut A
using aRawViewRepr
layout - RawView
MutTensor - a type alias for a
TensorBase
with an owned representation - RawView
Params - A type alias for the
ParamsBase
whose elements are of type*const A
using aRawViewRepr
layout - RawView
Tensor - A type alias for a
TensorBase
with a raw pointer representation. - Result
- a type alias for a Result configured with an
Error
as its error type. - Tensor
- a type alias for a
TensorBase
with an owned representation - Tensor
View - a type alias for a
TensorBase
with a view representation - Tensor
View Mut - a type alias for a
TensorBase
with a mutable view representation
Derive Macros§
- Keyed
- This macro generates a parameter struct and an enum of parameter keys.