Mind, Body, World- Foundations of Cognitive Science, 2013a

and then determine the relationship between the third note of the major chord
component and the fourth “added” note. This is particularly difficult because of the
pitch class representation, which throws away note-order information that might
be useful in identifying chord type.
It was decided that the network that would be trained on the chord classification
task would be a network of value units (Dawson & Schopflocher, 1992b). The
hidden units and output units in a network of value units use a Gaussian activation
function, which means that they behave as if they carve two parallel planes
through a pattern space. Such networks can be trained with a variation of the generalized
delta rule. This type of network was chosen for this problem for two reasons.
First, networks of value units have emergent properties that make them easier to
interpret than other types of networks trained on similar problems (Dawson, 2004;
Dawson et al., 1994). One reason for this is because value units behave as if they
are “tuned” to respond to very particular input signals. Second, previous research
on different versions of chord classification problems had produced networks that
revealed elegant internal structure (Yaremchuk & Dawson, 2005, 2008).
The simplest network of value units that could learn to solve the chord classification
problem required three hidden units. At the start of training, the value
of m for each unit was initialized as 0. (The value of m for a value unit is analogous
to a threshold in other types of units [Dawson, Kremer, & Gannon, 1994;
Dawson & Schopflocher, 1992b]; if a value unit’s net input is equal to m then the unit
generates a maximum activity of 1.00.) All connection weights were set to values randomly
selected from the range between –0.1 and 0.1. The network was trained with
a learning rate of 0.01 until it produced a “hit” for every output unit on every pattern.
Because of the continuous nature of the activation function, a hit was defined as follows:
a value of 0.9 or higher when the desired output was 1, and a value of 0.1 or lower
when the desired output was 0. The network that is interpreted below learned the
chord classification task after 299 presentations of the training set.
What is the role of a layer of hidden units? In a perceptron, which has no
hidden units, input patterns can only be represented in a pattern space. Recall
from the discussion of Figure 4-2 that a pattern space represents each pattern as
a point in space. The dimensionality of this space is equal to the number of input
units. The coordinates of each pattern’s point in this space are given by the activities
of the input units. For some networks, the positioning of the points in the
pattern space prevents some patterns from being correctly classified, because the
output units are unable to adequately carve the pattern space into the appropriate
decision regions.
In a multilayer perceptron, the hidden units serve to solve this problem. They
do so by transforming the pattern space into a hidden unit space (Dawson, 2004).
The dimensionality of a hidden unit space is equal to the number of hidden units
Elements of Connectionist Cognitive Science 165