Mind, Body, World- Foundations of Cognitive Science, 2013a

When New Connectionism arose in the 1980s, interest in it was fuelled by two
complementary perspectives (Medler, 1998). First, there was growing dissatisfaction
with the progress being made in classical cognitive science and symbolic artificial
intelligence (Dreyfus, 1992; Dreyfus & Dreyfus, 1988). Second, seminal introductions
to artificial neural networks (McClelland & Rumelhart, 1986; Rumelhart
& McClelland, 1986c) gave the sense that the connectionist architecture was a radical
alternative to its classical counterpart (Schneider, 1987).
The apparent differences between artificial neural networks and classical models
led to an early period of research in which networks were trained to accomplish tasks
that had typically been viewed as prototypical examples of classical cognitive science
(Bechtel, 1994; Rumelhart & McClelland, 1986a; Seidenberg & McClelland, 1989;
Sejnowski & Rosenberg, 1988). These networks were then used as “existence proofs”
to support the claim that non-classical models of classical phenomena are possible.
However, detailed analyses of these networks were not provided, which meant that,
apart from intuitions that connectionism is not classical, there was no evidence to
support claims about the non-classical nature of the networks’ solutions to the classical
problems. Because of this, this research perspective has been called gee whiz
connectionism (Dawson, 2004, 2009).
Of course, at around the same time, prominent classical researchers were criticizing
the computational power of connectionist networks (Fodor & Pylyshyn, 1988),
arguing that connectionism was a throwback to less powerful notions of associationism
that classical cognitive science had already vanquished (Bever, Fodor, &
Garrett, 1968; Chomsky, 1957, 1959b, 1965). Thus gee whiz connectionism served an
important purpose: providing empirical demonstrations that connectionism might
be a plausible medium in which cognitive science can be fruitfully pursued.
However, it was noted earlier that there exists a great deal of research on the
computational power of artificial neural networks (Girosi & Poggio, 1990; Hartman,
Keeler, & Kowalski, 1989; Lippmann, 1989; McCulloch & Pitts, 1943; Moody & Darken,
1989; Poggio & Girosi, 1990; Renals, 1989; Siegelmann, 1999; Siegelmann & Sontag,
1991); the conclusion from this research is that multilayered networks have the
same in-principle power as any universal machine. This leads, though, to the demise
of gee whiz connectionism, because if connectionist systems belong to the class of
universal machines, “it is neither interesting nor surprising to demonstrate that a
network can learn a task of interest” (Dawson, 2004, p. 118). If a network’s ability to
learn to perform a task is not of interest, then what is?
It can be extremely interesting, surprising, and informative to determine what regularities
the network exploits. What kinds of regularities in the input patterns has the
network discovered? How does it represent these regularities? How are these regularities
combined to govern the response of the network? (Dawson, 2004, p. 118)
Elements of Connectionist Cognitive Science 187