Mind, Body, World- Foundations of Cognitive Science, 2013a

input signals coming from all of the units. The connection weights used by the
hidden units place meaningful constraints on how these signals are combined.
Let us consider the role of the particular connection weights used by the hidden
units. Given the binary nature of the input encoding, the net input of any hidden
unit is simply the sum of the weights associated with each of the activated input
units. For a value unit, if the net input is equal to the value of the unit’s m then the
output generates a maximum value of 1.00. As the net input moves away from m in
either a positive or negative direction, activity quickly decreases. At the end of training,
the values of m for the three hidden units were 0.00, 0.00, and –0.03 for Hidden
Units 1, 2, and 3, respectively. Thus for each hidden unit, if the incoming signals are
essentially zero—that is if all the incoming signals cancel each other out—then high
activity will be produced.
Why then do Hidden Units 1 and 2 use the same set of four connection weights
but assign these weights to different sets of input notes? The answer is that these
hidden units capture similar chord relationships but do so using notes from different
strange circles.
This is shown by examining the responses of each hidden unit to each input
chord after training. Table 4-3 summarizes these responses, and shows that each
hidden unit generated identical responses to different subsets of input chords.
Input Chord
Activation
Chord Chord Root Hid1 Hid2 Hid3
Major
C, D, A, F#, G#, A# 0.16 0.06 0.16
C#, D#, F, G, A, B 0.06 0.16 0.16
Major7
C, D, A, F#, G#, A# 0.01 0.12 1.00
C#, D#, F, G, A, B 0.12 0.01 1.00
Dom7
C, D, A, F#, G#, A# 0.27 0.59 0.00
C#, D#, F, G, A, B 0.59 0.27 0.00
6th
C, D, A, F#, G#, A# 0.84 0.03 1.00
C#, D#, F, G, A, B 0.03 0.84 1.00
Table 4-3. The activations produced in each hidden unit by different subsets of
input chords.
From Table 4-3, one can see that the activity of Hidden Unit 3 is simplest to describe:
when presented with a dominant 7th chord, it produces an activation of 0 and a weak
activation to a major triad. When presented with either a major 7th or a 6th chord, it
produces maximum activity. This pattern of activation is easily explained by considering
the weights that feed into Hidden Unit 3 (Table 4-2). Any major 7th or 6th chord
is created out of two notes from one circle of major seconds and two notes from the
Elements of Connectionist Cognitive Science 169