Mind, Body, World- Foundations of Cognitive Science, 2013a

= 0.5
-2
1
1
1
= 1.5
1
p
q
Figure 4-4. A multilayer perceptron that can compute XOR.
The action of the hidden unit is crucial to the behaviour of the system. When neither
or only one of the input units activates, the hidden unit does not respond, so it sends
a signal of 0 to the output unit. As a result, in these three situations the output unit
turns on when either of the inputs is on (because the net input is over the threshold)
and turns off when neither input unit is on. When both input units are on, they send
an excitatory signal to the output unit. However, they also send a signal that turns
on the hidden unit, causing it to send inhibition to the output unit. In this situation,
the net input of the output unit is 1 + 1 – 2 = 0 which is below threshold, producing
zero output unit activity. The entire circuit therefore performs the XOR operation.
The behaviour of the Figure 4-4 multilayer perceptron can also be related to the
pattern space of Figure 4-2B. The lower cut in that pattern space is provided by the
output unit. The upper cut in that pattern space is provided by the hidden unit. The
coordination of the two units permits the circuit to solve this linearly nonseparable
problem.
Interpreting networks in terms of the manner in which they carve a pattern
space into decision regions suggests that learning can be described as determining
where cuts in a pattern space should be made. Any hidden or output unit that uses
a nonlinear, monotonic function like the Heaviside or the logistic can be viewed
as making a single cut in a space. The position and orientation of this cut is determined
by the weights of the connections feeding into the unit, as well as the threshold
or bias () of the unit. A learning rule modifies all of these components. (The
bias of a unit can be trained as if it were just another connection weight by assuming
that it is the signal coming from a special, extra input unit that is always turned on
[Dawson, 2004, 2005].)
146 Chapter 4