Mind, Body, World- Foundations of Cognitive Science, 2013a

structure/process distinction, because networks do not employ either explicit symbols
or rules. Instead, a network’s program is a set of causal or associative links
from signaling processors to receiving processors. The activity that is produced in
the receiving units is literally caused by having an input pattern of activity modulated
by an array of connection weights between units. In this sense, connectionist
models seem markedly associationist in nature (Bechtel, 1985); they can be comfortably
related to the old associationist psychology (Warren, 1921).
Artificial neural networks are not necessarily embodiments of empiricist philosophy.
Indeed, the earliest artificial neural networks did not learn from experience;
they were nativist in the sense that they had to have their connection weights
“hand wired” by a designer (McCulloch & Pitts, 1943). However, their associationist
characteristics resulted in a natural tendency for artificial neural networks to
become the face of modern empiricism. This is because associationism has always
been strongly linked to empiricism; empiricist philosophers invoked various
laws of association to explain how complex ideas could be constructed from the
knowledge provided by experience (Warren, 1921). By the late 1950s, when computers
were being used to bring networks to life, networks were explicitly linked
to empiricism (Rosenblatt, 1958). Rosenblatt’s artificial neural networks were not
hand wired. Instead, they learned from experience to set the values of their connection
weights.
What does it mean to say that artificial neural networks are empiricist? A famous
passage from Locke (1977, p. 54) highlights two key elements: “Let us then suppose
the mind to be, as we say, white paper, void of all characters, without any idea, how
comes it to be furnished? . . . To this I answer, in one word, from experience.”
The first element in the above quote is the “white paper,” often described as
the tabula rasa, or the blank slate: the notion of a mind being blank in the absence
of experience. Modern connectionist networks can be described as endorsing the
notion of the blank slate (Pinker, 2002). This is because prior to learning, the pattern
of connections in modern networks has no pre-existing structure. The networks
either start literally as blank slates, with all connection weights being equal to
zero (Anderson et al., 1977; Eich, 1982; Hinton & Anderson, 1981), or they start with
all connection weights being assigned small, randomly selected values (Rumelhart,
Hinton, & Williams, 1986a, 1986b).
The second element in Locke’s quote is that the source of ideas or knowledge or
structure is experience. Connectionist learning rules provide a modern embodiment
of this notion. Artificial neural networks are exposed to environmental stimulation—
activation of their input units—which results in changes to connection weights.
These changes furnish a network’s blank slate, resulting in a pattern of connectivity
that represents knowledge and implements a particular input-output mapping.
In some systems, called self-organizing networks, experience shapes connectivity
130 Chapter 4