Mind, Body, World- Foundations of Cognitive Science, 2013a

via unsupervised learning (Carpenter & Grossberg, 1992; Grossberg, 1980, 1987, 1988;
Kohonen, 1977, 1984). When learning is unsupervised, networks are only provided
with input patterns. They are not presented with desired outputs that are paired with
each input pattern. In unsupervised learning, each presented pattern causes activity
in output units; this activity is often further refined by a winner-take-all competition
in which one output unit wins the competition to be paired with the current input
pattern. Once the output unit is selected via internal network dynamics, its connection
weights, and possibly the weights of neighbouring output units, are updated via
a learning rule.
Networks whose connection weights are modified via unsupervised learning
develop sensitivity to statistical regularities in the inputs and organize their output
units to reflect these regularities. For instance, in a famous kind of self-organizing
network called a Kohonen network (Kohonen, 1984), output units are arranged in
a two-dimensional grid. Unsupervised learning causes the grid to organize itself
into a map that reveals the discovered structure of the inputs, where related patterns
produce neighbouring activity in the output map. For example, when such
networks are presented with musical inputs, they often produce output maps
that are organized according to the musical circle of fifths (Griffith & Todd, 1999;
Todd & Loy, 1991).
In cognitive science, most networks reported in the literature are not selforganizing
and are not structured via unsupervised learning. Instead, they are
networks that are instructed to mediate a desired input-output mapping. This is
accomplished via supervised learning. In supervised learning, it is assumed that the
network has an external teacher. The network is presented with an input pattern
and produces a response to it. The teacher compares the response generated by the
network to the desired response, usually by calculating the amount of error associated
with each output unit. The teacher then provides the error as feedback to the
network. A learning rule uses feedback about error to modify weights in such a way
that the next time this pattern is presented to the network, the amount of error that
it produces will be smaller.
A variety of learning rules, including the delta rule (Rosenblatt, 1958, 1962;
Stone, 1986; Widrow, 1962; Widrow & Hoff, 1960) and the generalized delta rule
(Rumelhart, Hinton, & Williams, 1986b), are supervised learning rules that work by
correcting network errors. (The generalized delta rule is perhaps the most popular
learning rule in modern connectionism, and is discussed in more detail in Section
4.9.) This kind of learning involves the repeated presentation of a number of inputoutput
pattern pairs, called a training set. Ideally, with enough presentations of a
training set, the amount of error produced to each member of the training set will
be negligible, and it can be said that the network has learned the desired inputoutput
mapping. Because these techniques require many presentations of a set of
Elements of Connectionist Cognitive Science 131