www.allitebooks.com

Chapter 8
Then we iterate over our testing dataset and add each as a sample into a new
SupervisedDataSet instance for testing. The code is as follows:
testing = SupervisedDataSet(X.shape[1], y.shape[1])
for i in range(X_test.shape[0]):
testing.addSample(X_test[i], y_test[i])
Now we can build a neural network. We will create a basic three-layer network that
consists of an input layer, an output layer, and a single hidden layer between them.
The number of neurons in the input and output layers is fixed. 400 features in our
dataset dictates that we need 400 neurons in the first layer, and 26 possible targets
dictate that we need 26 output neurons.
Determining the number of neurons in the hidden layers can be quite difficult.
Having too many results in a sparse network and means it is difficult to train
enough neurons to properly represent the data. This usually results in overfitting
the training data. If there are too few results in neurons that try to do too much
of the classification each and again don't train properly, underfitting the data is
the problem. I have found that creating a funnel shape, where the middle layer is
between the size of the inputs and the size of the outputs, is a good starting place.
For this chapter, we will use 100 neurons in the hidden layer, but playing with this
value may yield better results.
We import the buildNetwork function and tell it to build a network based on our
necessary dimensions. The first value, X.shape[1], is the number of neurons in the
input layer and it is set to the number of features (which is the number of columns in
X). The second feature is our decided value of 100 neurons in the hidden layer. The
third value is the number of outputs, which is based on the shape of the target array
y. Finally, we set network to use a bias neuron to each layer (except for the output
layer), effectively a neuron that always activates (but still has connections with a
weight that are trained). The code is as follows:
from pybrain.tools.shortcuts import buildNetwork
net = buildNetwork(X.shape[1], 100, y.shape[1], bias=True)
From here, we can now train the network and determine good values for the
weights. But how do we train a neural network?
Back propagation
The back propagation (backprop) algorithm is a way of assigning blame to each
neuron for incorrect predictions. Starting from the output layer, we compute which
neurons were incorrect in their prediction, and adjust the weights into those neurons
by a small amount to attempt to fix the incorrect prediction.
[ 173 ]