<!DOCTYPE HTML>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<base href="">
<link rel="stylesheet" href="book.css">
<link href='http://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800' rel='stylesheet' type='text/css'>
<link rel="shortcut icon" href="favicon.png">
<!-- Font Awesome -->
<link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css">
<link rel="stylesheet" href="highlight.css">
<link rel="stylesheet" href="tomorrow-night.css">
<!-- MathJax -->
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<!-- Fetch JQuery from CDN but have a local fallback -->
<script src="http://code.jquery.com/jquery-2.1.4.min.js"></script>
<script>
if (typeof jQuery == 'undefined') {
document.write(unescape("%3Cscript src='jquery.js'%3E%3C/script%3E"));
}
</script>
</head>
<body>
<!-- Set the theme before any content is loaded, prevents flash -->
<script type="text/javascript">
var theme = localStorage.getItem('theme');
if (theme == null) { theme = 'light'; }
$('body').removeClass().addClass(theme);
</script>
<!-- Hide / unhide sidebar before it is displayed -->
<script type="text/javascript">
var sidebar = localStorage.getItem('sidebar');
if (sidebar === "hidden") { $("html").addClass("sidebar-hidden") }
else if (sidebar === "visible") { $("html").addClass("sidebar-visible") }
</script>
<div id="sidebar" class="sidebar">
<ul class="chapter"><li><a href="./leaf.html"><strong>1.</strong> Leaf</a></li><li><a href="./layers.html"><strong>2.</strong> Layers</a></li><li><ul class="section"><li><a href="./layer-lifecycle.html"><strong>2.1.</strong> Layer Lifecycle</a></li><li><a href="./building-networks.html" class="active"><strong>2.2.</strong> Create a Network</a></li><li><a href="./create-new-layer.html"><strong>2.3.</strong> Create a new Layer</a></li></ul></li><li><a href="./solvers.html"><strong>3.</strong> Solvers</a></li><li><ul class="section"><li><a href="./optimize-layers.html"><strong>3.1.</strong> Optimize Layers</a></li><li><a href="./multi-device-optimization.html"><strong>3.2.</strong> Multi-Device Optimization</a></li><li><a href="./distributed-optimization.html"><strong>3.3.</strong> Distributed Optimization</a></li></ul></li><li><a href="./backend.html"><strong>4.</strong> Backend</a></li><li><a href="./deep-learning-glossary.html"><strong>5.</strong> Glossary</a></li><li class="spacer"></li><li><a href="http://autumnai.github.io/leaf/leaf/index.html"><strong>6.</strong> Rust API Documentation</a></li></ul>
</div>
<div id="page-wrapper" class="page-wrapper">
<div class="page">
<div id="menu-bar" class="menu-bar">
<div class="left-buttons">
<i id="sidebar-toggle" class="fa fa-bars"></i>
<i id="theme-toggle" class="fa fa-paint-brush"></i>
</div>
<h1 class="menu-title"></h1>
<div class="right-buttons">
<i id="print-button" class="fa fa-print" title="Print this book"></i>
</div>
</div>
<div id="content" class="content">
<h1>Create a Network</h1>
<p>In the previous chapters, we learned that in Leaf everything is build by
layers and that the constructed thing is again a layer, which means it can
function as a new building block for something bigger. This is possible, because
a <code>Layer</code> can implement any behavior as long as it takes an input and produces
an output.</p>
<p>In <a href="./layer-lifecycle.html">2.1 Layer Lifecycle</a>
we have seen, that only one <code>LayerConfig</code> can be used to turn it via
<code>Layer::from_config</code> into an actual <code>Layer</code>. But as Deep Learning relies on
chaining multiple layers together, we need a <code>Layer</code>, who implements this
behavior for us.</p>
<p>Enter the container layers.</p>
<h3>Networks via the <code>Sequential</code> layer</h3>
<p>A <code>Sequential</code> Layer is a layer of type container layer. The config of a
container layer has a special method called,
<code>.add_layer</code> which takes one <code>LayerConfig</code> and adds it to an ordered list in the
<code>SequentialConfig</code>.</p>
<p>When turning a <code>SequentialConfig</code> into a <code>Layer</code> by passing the config to
<code>Layer::from_config</code>, the behavior of the <code>Sequential</code> is to initialize all the
layers which were added via <code>.add_layer</code> and connect the layers with each other.
This means, the output of one layer becomes the input of the next layer in the
list.</p>
<p>The input of a sequential <code>Layer</code> becomes the input of the
first layer in the sequential worker, the sequential worker then takes care
of passing the input through all the layers and the output of the last layer
then becomes the output of the <code>Layer</code> with the sequential worker. Therefore
a sequential <code>Layer</code> fulfills the requirements of a <code>Layer</code> - take an input,
return an output.</p>
<pre><code class="language-rust">// short form for: &LayerConfig::new("net", LayerType::Sequential(cfg))
let mut net_cfg = SequentialConfig::default();
net_cfg.add_input("data", &vec![batch_size, 28, 28]);
net_cfg.add_layer(LayerConfig::new("reshape", ReshapeConfig::of_shape(&vec![batch_size, 1, 28, 28])));
net_cfg.add_layer(LayerConfig::new("conv", ConvolutionConfig { num_output: 20, filter_shape: vec![5], stride: vec![1], padding: vec![0] }));
net_cfg.add_layer(LayerConfig::new("pooling", PoolingConfig { mode: PoolingMode::Max, filter_shape: vec![2], stride: vec![2], padding: vec![0] }));
net_cfg.add_layer(LayerConfig::new("linear1", LinearConfig { output_size: 500 }));
net_cfg.add_layer(LayerConfig::new("sigmoid", LayerType::Sigmoid));
net_cfg.add_layer(LayerConfig::new("linear2", LinearConfig { output_size: 10 }));
net_cfg.add_layer(LayerConfig::new("log_softmax", LayerType::LogSoftmax));
// set up the sequential layer aka. a deep, convolutional network
let mut net = Layer::from_config(backend.clone(), &net_cfg);
</code></pre>
<p>As a sequential layer is like any other layer, we can use sequential layers as
building blocks for larger networks. Important building blocks of a network can
be grouped into a sequential layer and published as a crate for others to use.</p>
<pre><code class="language-rust">// short form for: &LayerConfig::new("net", LayerType::Sequential(cfg))
let mut conv_net = SequentialConfig::default();
conv_net.add_input("data", &vec![batch_size, 28, 28]);
conv_net.add_layer(LayerConfig::new("reshape", ReshapeConfig::of_shape(&vec![batch_size, 1, 28, 28])));
conv_net.add_layer(LayerConfig::new("conv", ConvolutionConfig { num_output: 20, filter_shape: vec![5], stride: vec![1], padding: vec![0] }));
conv_net.add_layer(LayerConfig::new("pooling", PoolingConfig { mode: PoolingMode::Max, filter_shape: vec![2], stride: vec![2], padding: vec![0] }));
conv_net.add_layer(LayerConfig::new("linear1", LinearConfig { output_size: 500 }));
conv_net.add_layer(LayerConfig::new("sigmoid", LayerType::Sigmoid));
conv_net.add_layer(LayerConfig::new("linear2", LinearConfig { output_size: 10 }));
let mut net_cfg = SequentialConfig::default();
net_cfg.add_layer(conv_net);
net_cfg.add_layer(LayerConfig::new("linear", LinearConfig { output_size: 500 }));
net_cfg.add_layer(LayerConfig::new("log_softmax", LayerType::LogSoftmax));
// set up the 'big' network
let mut net = Layer::from_config(backend.clone(), &net_cfg);
</code></pre>
<h3>Networks via other container layers</h3>
<p>So far, there is only the sequential layer, but other container layers, with
slightly different behaviors are conceivable. For example a parallel or
concat layer in addition to the sequential layer.</p>
<p>How to 'train' or optimize the constructed network is topic of chapter <a href="./solvers.html">3.
Solvers</a></p>
</div>
<!-- Mobile navigation buttons -->
<a href="./layer-lifecycle.html" class="mobile-nav-chapters previous">
<i class="fa fa-angle-left"></i>
</a>
<a href="./create-new-layer.html" class="mobile-nav-chapters next">
<i class="fa fa-angle-right"></i>
</a>
</div>
<a href="./layer-lifecycle.html" class="nav-chapters previous" title="You can navigate through the chapters using the arrow keys">
<i class="fa fa-angle-left"></i>
</a>
<a href="./create-new-layer.html" class="nav-chapters next" title="You can navigate through the chapters using the arrow keys">
<i class="fa fa-angle-right"></i>
</a>
</div>
<!-- Local fallback for Font Awesome -->
<script>
if ($(".fa").css("font-family") !== "FontAwesome") {
$('<link rel="stylesheet" type="text/css" href="_FontAwesome/css/font-awesome.css">').prependTo('head');
}
</script>
<script src="highlight.js"></script>
<script src="book.js"></script>
</body>
</html>
