Mind, Body, World- Foundations of Cognitive Science, 2013a

input pattern in an all-or-none fashion to a particular category. A second is function
approximation: generating a continuous response to a set of input values.
Section 4.6 then proceeds to computational analyses of how capable networks
are of accomplishing these tasks. These analyses prove that networks are as powerful
as need be, provided that they include hidden units. They can serve as arbitrary
pattern classifiers, meaning that they can solve any pattern classification problem
with which they are faced. They can also serve as universal function approximators,
meaning that they can fit any continuous function to an arbitrary degree of
precision. This computational power suggests that artificial neural networks belong
to the class of universal machines. The section ends with a brief review of computational
analyses, which conclude that connectionist networks indeed can serve as
universal Turing machines and are therefore computationally sophisticated enough
to serve as plausible models for cognitive science.
Computational analyses need not limit themselves to considering the general
power of artificial neural networks. Computational analyses can be used to explore
more specific questions about networks. This is illustrated in Section 4.7, “What
Do Output Unit Activities Represent?” in which we use formal methods to answer
the question that serves as the section’s title. The section begins with a general discussion
of theories that view biological agents as intuitive statisticians who infer
the probability that certain events may occur in the world (Peterson & Beach, 1967;
Rescorla, 1967, 1968). An empirical result is reviewed that suggests artificial neural
networks are also intuitive statisticians, in the sense that the activity of an output
unit matches the probability that a network will be “rewarded” (i.e., trained to turn
on) when presented with a particular set of cues (Dawson et al., 2009).
The section then ends by providing an example computational analysis: a formal
proof that output unit activity can indeed literally be interpreted as a conditional
probability. This proof takes advantage of known formal relations between neural networks
and the Rescorla-Wagner learning rule (Dawson, 2008; Gluck & Bower, 1988;
Sutton & Barto, 1981), as well as known formal relations between the Rescorla-
Wagner learning rule and contingency theory (Chapman & Robbins, 1990).
4.6 Beyond the Terminal Meta-postulate
Connectionist networks are associationist devices that map inputs to outputs, systems
that convert stimuli into responses. However, we saw in Chapter 3 that classical
cognitive scientists had established that the stimulus-response theories of
behaviourist psychology could not adequately deal with the recursive structure of
natural language (Chomsky, 1957, 1959b, 1965, 1966). In the terminal meta-postulate
argument (Bever, Fodor, and Garrett, 1968), it was noted that the rules of associative
theory defined a “terminal vocabulary of a theory, i.e., over the vocabulary in which
Elements of Connectionist Cognitive Science 149