Chapter 2. Prehension

394 A pp e It d ic e s
training set; if they are the same, the network has learned the correct
computation for that input. If they are different, the second phase
occurs. During the second phase, the weights between the neurons
are adjusted by propagating the error backwards using equation (13) to
change the weights.
Weight space refers to the space of all possible weight
combinations. It is generally true that the more weights available to
adjust (Le., the higher the weight space dimensionality), the more
likely it is that a solution will be found. As a simple example consider
the simple mapping shown in the table in Figure C.7. Four patterns
are mapped into a single value, computing an Exclusive-OR function.
A binary representation is shown at the left. A ‘distributed’
representation is also shown that encodes the same information in a
four element, rather than two element, vector. For each encoding, a
network is shown in Figure C.7. For the binary representation, there
are no connection weights such that a single neuron can achieve the
mapping. However, with the distributed representation, it is a simple
matter to find a set of weights which solve the problem. There are
several arguments why this strategy tends to be successful, but it
suffices to say that mappings are more likely to be realized in a higher
dimensional space.
Input output
Binarv Distributed
0 0- 0 0 0 1 0
0 1 0 0 1 0 1
1 0 0 1 0 0 1
1 1 1 0 0 0 0
Binary
3
Distributed
Figure C.7 Two encodings of the same function.