Mind, Body, World- Foundations of Cognitive Science, 2013a

in the layer. Patterns are again represented as points in this space; however, in this
space their coordinates are determined by the activities they produce in each hidden
unit. The hidden unit space is a transformation of the pattern space that involves
detecting higher-order features. This usually produces a change in dimensionality—the
hidden unit space often has a different number of dimensions than does
the pattern space—and a repositioning of the points in the new space. As a result,
the output units are able to carve the hidden unit space into a set of decision regions
that permit all of the patterns, repositioned in the hidden unit space, to be correctly
classified.
This account of the role of hidden units indicates that the interpretation of the
internal structure of a multilayer perceptron involves answering two different questions.
First, what kinds of features are the hidden units detecting in order to map patterns
from the pattern space into the hidden unit space? Second, how do the output
units process the hidden unit space to solve the problem of interest? The chord classification
network can be used to illustrate how both questions can be addressed.
First, when mapping the input patterns into the hidden unit space, the hidden
units must be detecting some sorts of musical regularities. One clue as to what these
regularities may be is provided by simply examining the connection weights that
feed into them, provided in Table 4-2.
Input
Note
Hidden 1 Hidden 1
Class
Hidden 2 Hidden 2
Class
B
D#
0.53
0.53
Circle of Major
0.12
0.12
Circle of Major
0.75
0.75
G 0.53
Thirds 1
0.12
Thirds 1
0.75
A –0.53
Circle of Major
–0.12
Circle of Major
0.75
C# –0.53 –0.12 0.75
F –0.53
Thirds 2
–0.12
Thirds 2
0.75
C 0.12
Circle of Major
–0.53
Circle of Major
–0.77
G# 0.12 –0.53 –0.77
E 0.12
Thirds 3
–0.53
Thirds 3
–0.77
F# –0.12
Circle of Major
0.53
Circle of Major
–0.77
A# –0.12 0.53 –0.77
D –0.12
Thirds 4
0.53
Thirds 4
–0.77
Hidden 3 Hidden 3
Class
Circle of
Major
Seconds 1
Circle of
Major
Seconds 2
Table 4-2. Connection weights from the 12 input units to each of the three
hidden units. Note that the first two hidden units adopt weights that assign
input notes to the four circles of major thirds. The third hidden unit adopts
weights that assign input notes to the two circles of major seconds.
166 Chapter 4