pdfcoffee

Chapter 1
In Figure 4 each node in the first hidden layer receives an input and "fires" (0,1)
according to the values of the associated linear function. Then, the output of the first
hidden layer is passed to the second layer where another linear function is applied,
the results of which are passed to the final output layer consisting of one single
neuron. It is interesting to note that this layered organization vaguely resembles
the organization of the human vision system, as we discussed earlier.
Problems in training the perceptron and their
solutions
Let's consider a single neuron; what are the best choices for the weight w and the bias
b? Ideally, we would like to provide a set of training examples and let the computer
adjust the weight and the bias in such a way that the errors produced in the output
are minimized.
In order to make this a bit more concrete, let's suppose that we have a set of images
of cats and another separate set of images not containing cats. Suppose that each
neuron receives input from the value of a single pixel in the images. While the
computer processes those images, we would like our neuron to adjust its weights
and its bias so that we have fewer and fewer images wrongly recognized.
This approach seems very intuitive, but it requires a small change in the weights (or
the bias) to cause only a small change in the outputs. Think about it: if we have a
big output jump, we cannot learn progressively. After all, kids learn little by little.
Unfortunately, the perceptron does not show this "little-by-little" behavior. A
perceptron is either a 0 or 1, and that's a big jump that will not help in learning (see
Figure 5):
Figure 5: Example of perceptron - either a 0 or 1
[ 9 ]