collenchyma-nn 0.3.4

collenchyma plugin providing Neural Network operations

Crate collenchyma_nn [] [src]

Provides a Collenchyma Plugin, to extend Collenchyma with Neural Network related operations such as convolutions, pooling, ReLU, etc. A full list of operations provided by this Plugin, can be found at the provided Operations section.

Overview

This Collenchyma Plugin extends Collenchyma's Backend with NN related methods/operations. This allows you to run, these operations (and therefore your application) on your local machine as well as on servers, mobiles or any other machine (as if they were written for common CPU execution), while receiving the significant performance increases (usually one-to-two orders of magnitutde), by executing the operations on special purpose hardware such as GPUs - if they are available. Usage examples can be found in the next section.

The architecture of a Plugin is quite easy. It defines one Plugin Trait, in this case the NN trait, which implements basic functionality for initialization and multiple Plugin Operation Traits which define the methods which are going to be available on the Backed, as the Plugin Trait as well as the Plugin Operations Traits are implemented for the Collenchyma Backends (CUDA, OpenCL, Native). The operations take as arguments one or many SharedTensors, holding the data over which the operation should happen, and none or one Operation Configuration.

Usage

An example on how to write some data into a SharedTensor and compute the result of the sigmoid function for each value:

extern crate collenchyma as co;
extern crate collenchyma_nn as nn;
use co::prelude::*;
use nn::*;

fn write_to_memory<T: Copy>(mem: &mut MemoryType, data: &[T]) {
    if let &mut MemoryType::Native(ref mut mem) = mem {
        let mut mem_buffer = mem.as_mut_slice::<T>();
        for (index, datum) in data.iter().enumerate() {
            mem_buffer[index] = *datum;
        }
    }
}

pub fn main() {
    // Initialize a CUDA Backend.
    // Usually you would not use CUDA but let Collenchyma pick what is available on the machine.
    let backend = Backend::<Cuda>::default().unwrap();
    // Initialize two SharedTensors.
    let mut x = SharedTensor::<f32>::new(backend.device(), &(1, 1, 3)).unwrap();
    let mut result = SharedTensor::<f32>::new(backend.device(), &(1, 1, 3)).unwrap();
    // Fill `x` with some data.
    let payload: &[f32] = &::std::iter::repeat(1f32).take(x.capacity()).collect::<Vec<f32>>();
    let native = Native::new();
    let cpu = native.new_device(native.hardwares()).unwrap();
    x.add_device(&cpu).unwrap(); // Add native host memory
    x.sync(&cpu).unwrap(); // Sync to native host memory
    write_to_memory(x.get_mut(&cpu).unwrap(), payload); // Write to native host memory.
    x.sync(backend.device()).unwrap(); // Sync the data to the CUDA device.
    // Run the sigmoid operation, provided by the NN Plugin, on your CUDA enabled GPU.
    backend.sigmoid(&mut x, &mut result).unwrap();
    // See the result.
    result.add_device(&cpu).unwrap(); // Add native host memory
    result.sync(&cpu).unwrap(); // Sync the result to host memory.
    println!("{:?}", result.get(&cpu).unwrap().as_native().unwrap().as_slice::<f64>());
}

Provided Operations

This Plugins provides the following operations. (Forward + Backward) A - means not yet implemented.

Operation CUDA OpenCL Native
Sigmoid cuDNN v3 - Rust
SigmoidPointwise cuDNN v3 -
ReLU cuDNN v3 - Rust
ReLUPointwise cuDNN v3 -
Tanh cudNN v3 - Rust
TanhPointwise cuDNN v3 -
Normalization (LRN) cudNN v3 - -
Convolution cudNN v3 - -
Softmax cudNN v3 - Rust
LogSoftmax cudNN v3 - Rust
Pooling Max cudNN v3 - -
Pooling Avg cudNN v3 - -

Modules

frameworks

Provides the specific Framework implementations for the Library Operations.

Macros

impl_oconf_for_cc!
impl_oconf_for_clrn!
impl_oconf_for_pooling!
impl_ops_convolution_for!
impl_ops_convolution_for!
impl_ops_log_softmax_for!
impl_ops_log_softmax_for!
impl_ops_lrn_for!
impl_ops_lrn_for!
impl_ops_pooling_for!
impl_ops_pooling_for!
impl_ops_relu_for!
impl_ops_relu_for!
impl_ops_relu_pointwise_for!
impl_ops_sigmoid_for!
impl_ops_sigmoid_for!
impl_ops_sigmoid_pointwise_for!
impl_ops_softmax_for!
impl_ops_softmax_for!
impl_ops_tanh_for!
impl_ops_tanh_for!
impl_ops_tanh_pointwise_for!

Enums

ConvBackwardDataAlgo

Different algorithms to compute the gradient with respect to the filter.

ConvBackwardFilterAlgo

Different algorithms to compute the gradient with respect to the filter.

ConvForwardAlgo

Different algorithms to compute the convolution forward algorithm.

Traits

Convolution

Provides the functionality for a Backend to support Convolution operations.

ConvolutionConfig

Provides Convolution Config functionality.

LRN

Provides the functionality for a Backend to support Local Response Normalization operations.

LogSoftmax

Provides the functionality for a Backend to support LogSoftmax operations.

NN

Provides the functionality for a backend to support Neural Network related operations.

NNOperationConfig

Provides generic NN Operation Config functionality.

Pooling

Provides the functionality for a Backend to support Pooling operations.

Relu

Provides the functionality for a Backend to support ReLU operations.

ReluPointwise

Provides the functionality for pointwise ReLU operations (overwrites the input with the result of the operation).

Sigmoid

Provides the functionality for a Backend to support Sigmoid operations.

SigmoidPointwise

Provides the functionality for pointwise Sigmoid operations (overwrites the input with the result of the operation).

Softmax

Provides the functionality for a Backend to support Softmax operations.

Tanh

Provides the functionality for a Backend to support TanH operations.

TanhPointwise

Provides the functionality for pointwise ReLU operations (overwrites the input with the result of the operation).