Crate concision_neural

Source
Expand description

Various components, implementations, and traits for creating neural networks. The crate builds off of the concision_core crate, making extensive use of the ParamsBase type to define the parameters of layers within a network.

§Overview

Neural networks are a fundamental part of machine learning, and this crate provides a comprehensive set of tools to build, configure, and train neural network models. Listed below are several key components of the crate:

  • Model: A trait for defining a neural network model.
  • StandardModelConfig: A standard configuration for the models
  • ModelFeatures: A default implementation of the ModelLayout trait that sufficiently defines both shallow and deep neural networks.

§Model Parameters

Additionally, the crate defines a sequential

Note: You should stick with the type aliases for the ModelParamsBase type, as they drastically simplify the type-face of the model parameters. Attempting to generalize over the hidden layers of the model might lead to excessive complexity. That being said, there are provided methods and routines to convert from a shallow to deep model, and vice versa.

§Traits

This crate extends the Forward and Backward traits from the core crate to provide additional functionality for neural networks.

  • Predict: A more robust implementation of the [Forward] trait
  • Train: A trait for training a neural network model.

Modules§

config
error
this module defines the ModelError type, used to define the various errors encountered by the different components of a neural network. Additionally, the ModelResult alias is defined for convenience, allowing for a more ergonomic way to handle results that may fail.
layers
This module implments various layers for a neural network
layout
params
this module provides the ModelParamsBase type and its associated aliases. The implementation focuses on providing a generic container for the parameters of a neural network.
train
This module focuses on developing lazy trainers for neural networks.

Structs§

Dropout
The Dropout layer is randomly zeroizes inputs with a given probability (p). This regularization technique is often used to prevent overfitting.
HyperparametersIter
An iterator over the variants of Hyperparameters
KeyValue
The KeyValue type is used to generically represent a simple key-value pair within a store.
LayerBase
The LayerBase struct is a base representation of a neural network layer, essentially binding an activation function, F, to a set of parameters, ParamsBase<S, D>.
ModelFeatures
The ModelFeatures provides a common way of defining the layout of a model. This is used to define the number of input features, the number of hidden layers, the number of hidden features, and the number of output features.
ModelParamsBase
The ModelParamsBase object is a generic container for storing the parameters of a neural network, regardless of the layout (e.g. shallow or deep). This is made possible through the introduction of a generic hidden layer type, H, that allows us to define aliases and additional traits for contraining the hidden layer type. That being said, we don’t reccoment using this type directly, but rather use the provided type aliases such as [DeepModelParams] or [ShallowModelParams] or their owned variants. These provide a much more straighforward interface for typing the parameters of a neural network. We aren’t too worried about the transmutation between the two since users desiring this ability should simply stick with a deep representation, initializing only a single layer within the respective container.
StandardModelConfig
Trainer

Enums§

HyperParams
Hyperparameters
Auto-generated discriminant enum variants
ModelError
The ModelError type is used to define the various errors encountered by the different components of a neural network. It is designed to be comprehensive, covering a wide range of potential issues that may arise during the operation of neural network components, such as invalid configurations, training failures, and other runtime errors. This error type is intended to provide a clear and consistent way to handle errors across the neural network components, making it easier to debug and resolve issues that may occur during the development and execution of neural network models.
ModelFormat
The ModelFormat type enumerates the various formats a neural network may take, either shallow or deep, providing a unified interface for accessing the number of hidden features and layers in the model. This is done largely for simplicity, as it eliminates the need to define a particular type of network as its composition has little impact on the actual requirements / algorithms used to train or evaluate the model (that is, outside of the obvious need to account for additional hidden layers in deep configurations). In other words, both shallow and deep networks are requried to implement the same traits and fulfill the same requirements, so it makes sense to treat them as a single type with different configurations. The differences between the networks are largely left to the developer and their choice of activation functions, optimizers, and other considerations.

Traits§

DeepModelRepr
The DeepModelRepr trait for deep neural networks
DeepNeuralNetwork
The DeepNeuralNetwork trait is a specialization of the Model trait that provides additional functionality for deep neural networks. This trait is
Layer
A generic trait defining the composition of a layer within a neural network.
Model
The Model trait defines the core interface for all models; implementors will need to provide the type of configuration used by the model, the type of layout used by the model, and the type of parameters used by the model. The crate provides standard, or default, definitions of both the configuration and layout types, however, for
ModelExt
ModelLayout
The ModelLayout trait defines an interface for object capable of representing the layout; i.e. the number of input, hidden, and output features of a neural network model containing some number of hidden layers.
ModelTrainer
NetworkConfig
The NetworkConfig trait extends the RawConfig trait to provide a more robust interface for neural network configurations.
Predict
The Predict trait is designed as a model-specific interface for making predictions. In the future, we may consider opening the trait up allowing for an alternative implementation of the trait, but for now, it is simply implemented for all implementors of the Forward trait.
PredictWithConfidence
The PredictWithConfidence trait is an extension of the Predict trait, providing an additional method to obtain predictions along with a confidence score.
RawConfig
The RawConfig trait defines a basic interface for all configurations used within the framework for neural networks, their layers, and more.
RawHidden
The RawHidden trait for compatible representations of hidden layers
ShallowModelRepr
The ShallowModelRepr trait for shallow neural networks
Train
This trait defines the training process for the network
TrainingConfiguration

Functions§

_verify_input_and_hidden_shape
verify if the input and hidden dimensions are compatible by checking:

Type Aliases§

DeepModelParams
a type alias for an owned representation of the DeepParamsBase generic of type A and the dimension D.
DeepParamsBase
a type alias for a deep representation of the ModelParamsBase using a vector of parameters as the hidden layers.
ModelParams
A type alias for an owned representation of the ModelParamsBase generic of type A and the dimension D.
ModelResult
a type alias for a Result configured to use the ModelError implementation as its error type.
ShallowModelParams
a type alias for an owned representation of the DeepParamsBase generic of type A and the dimension D.
ShallowParamsBase
a type alias for a shallow representation of the ModelParamsBase using a single ParamsBase instance as the hidden layer.