www.allitebooks.com

Chapter 11
Each of these issues has been addressed in recent times, leading to a resurgence
in popularity of neural networks. Computational power is now much more easily
available than 30 years ago, and advances in algorithms for training mean that we
can now readily use that power.
Intuition
The aspect that differentiates deep neural networks from the more basic neural
network we saw in Chapter 8, Beating CAPTCHAs with Neural Networks, is size.
A neural network is considered deep when it has two or more hidden layers. In
practice, a deep neural network is often much larger, both in the number of nodes
in each layer and also the number of layers. While some of the research of the mid-
2000s focused on very large numbers of layers, smarter algorithms are reducing the
actual number of layers needed.
A neural network basically takes very basic features as inputs—in the case of
computer vision, it is simple pixel values. Then, as that data is combined and pushed
through the network, these basic features combine into more complex features.
Sometimes, these features have little meaning to humans, but they represent the
aspects of the sample that the computer looks for to make its classification.
Implementation
Implementing these deep neural networks can be quite challenging due to their size.
A bad implementation will take significantly longer to run than a good one, and may
not even run at all due to memory usage.
A basic implementation of a neural network might start by creating a node
class and collecting a set of these into a layer class. Each node is then connected
to a node in the next layer using an instance of an Edge class. This type of
implementation, a class-based one, is good to show how networks work,
but is too inefficient for larger networks.
Neural networks are, at their core, simply mathematical expressions on matrices.
The weights of the connections between one network and the next can be
represented as a matrix of values, where the rows represent nodes in the first layer
and the columns represent the nodes in the second layer (the transpose of this matrix
is used sometimes too). The value is the weight of the edge between one layer and
the next. A network can then be defined as a set of these weight matrices. In addition
to the nodes, we add a bias term to each layer, which is basically a node that is
always on and connected to each neuron in the next layer.
[ 247 ]