1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
//! Leaf is a open, modular and clear-designed Machine Intelligence Framework providing
//! state-of-the-art performance for distributed (Deep|Machine) Learning - sharing concepts from
//! Tensorflow and Caffe.
//!
//! An important module in Leaf is the backend-agnostic, high-performance computation Framework
//! [Collenchyma][collenchyma], which combines performance and usability for Leaf Networks.
//! This allows you to run and deploy Leaf Networks to servers, desktops or even mobiles
//! using the full available computation power of GPUs or other CUDA/OpenCL supported
//! devices for the learning of your Networks. And if your machine does not have a GPU or you do
//! not want to install CUDA/OpenCL on your local machine, Leaf will gracefully fall back to
//! your native host CPU.
//!
//! ## Architecture
//!
//! Leaf's Networks are a compositional model, representing a collection of connected
//! [layers][layers], making operations over numerical data.
//!
//! Layers, the building block of a Leaf Network, are small units, describing computation over
//! numerical input data. Generally speaking Layers take input and produce an output, but
//! essentially a Layer can describe any functionality e.g. logging as long as it obeys to the
//! general behaviour specifications of a Layer. A Layer can be grouped in one of four
//! Layer types which are closer defined at the [Layers page][layers]. Every
//! layer serves a special purpose and can occur zero, one or many times inside a Network.
//!
//! Leaf uses [Collenchymas'][collenchyma] SharedTensor, an N-dimensional array
//! for a unified memory interface over the actual data for automatic synchronization between
//! different devices (CUDA, OpenCL, host CPU). A SharedTensor stores the actual data flowing
//! through the system and the weights required for some Layers.
//! The data in a SharedTensor can be copied from backend to backend
//! and can be used for computations on CUDA, OpenCL and native host CPU.
//! It provides performance optimizations and automatically takes care of memory management and synchronization.
//!
//! A neural network can be created by combining container layers like the `Sequential` Layer.
//! Those can be nested and allow for bigger neural networks to be constructed while still
//! retaining the interface of a Layer.
//!
//! The learning and optimization of the Network happens at the [Solver][solver] and is decoupled
//! from the network making the setup clean and flexible. One of the four layer types is a Loss
//! Layer, which is used for the interaction of Network and Solver. The Network produces the loss
//! and gradients, which the Solver uses to optimize the Network through parameter updates. Beside
//! that, the Solver provides housekeeping and other evaluations of the Network. All operation
//! on the Solver happen through Collenchyma, therefore can be executed on Cuda, OpenCL or native
//! host CPU as well.
//!
//! Leaf provides a robust and modular design, which allows to express almost any numerical
//! computation including SVMs, RNNs and other popular learning algorithms. We hope that Leaf can
//! help future research and production development alike as it combines expressiveness,
//! performance and usability.
//!
//! ## Philosophy
//!
//! We are strong believers in the technology of Machine Learning.
//!
//! We put our experience in software engineering into Leaf, to solve our own
//! need for a modern, performant and easy-to-use Deep Learning Framework.
//! These principles direct our decisions on Leaf and related projects.
//!
//! * __Cutting Edge Performance__:</br>
//! For research and industry speed and efficency are crucial for
//! state-of-the-art machine learning over massive data and networks.
//! * __Open and Expressive Architecture__:</br>
//! Designing an open architecture that follows best practices and concepts in
//! Engineering such as modularity, flexibility and expressiveness is critical
//! to stimulate future innovation.
//! * __Clear and Transparent Documentation__:</br>
//! A well-written documentation that addresses both concepts and
//! implementations, empowers developers and researchers to contribute their
//! unique experience to the project for the benefit of everyone.
//!
//! ## Development
//!
//! The implementation of various Layers is pretty scarce at the moment.<br/>
//! There are around a dozen layers, which are really important and would
//! increase the value and functionality of Leaf tremendously.<br/>
//! Progress is tracked at<br/>
//!
//! - [Issue #18 for Loss Layers][issue-loss]
//! - [Issue #19 for Activation Layers][issue-activation]
//! - [Issue #20 for Common Layers][issue-common]
//!
//! [collenchyma]: https://github.com/autumnai/collenchyma
//! [network]: ./network/index.html
//! [layers]: ./layers/index.html
//! [activation]: ./layers/activation/index.html
//! [loss]: ./layers/loss/index.html
//! [solvers]: ./solvers/index.html
//! [optimization]: https://en.wikipedia.org/wiki/Stochastic_optimization
//!
//! [issue-loss]: https://github.com/autumnai/leaf/issues/18
//! [issue-activation]: https://github.com/autumnai/leaf/issues/19
//! [issue-common]: https://github.com/autumnai/leaf/issues/20
// used when run with cargo test --no-run --features clippy
// or cargo build --features clippy
extern crate timeit;
extern crate log;
extern crate rand;
extern crate num;
extern crate capnp;
extern crate collenchyma as co;
extern crate collenchyma_blas as coblas;
extern crate collenchyma_nn as conn;
// include capnp code generated by `build.rs`