Mind, Body, World- Foundations of Cognitive Science, 2013a

The ability of A’s later activity to reproduce B is due to these modified connections
between the two sets of neurons.
The thought of A must awaken that of B, because a, b, c, d, e, will each and all
discharge into l through the paths by which their original discharge took place.
Similarly they will discharge into m, n, o, and p; and these latter tracts will also
each reinforce the other’s action because, in the experience B, they have already
vibrated in unison. (James, 1890a, p. 569)
James’ (1890a) biological account of association reveals three properties that are
common to modern connectionist networks. First, his system is parallel: more than
one neuron can be operating at the same time. Second, his system is convergent:
the activity of one of the output neurons depends upon receiving or summing the
signals sent by multiple input neurons. Third, his system is distributed: the association
between A and B is the set of states of the many “tracts” illustrated in Figure 4-1;
there is not just a single associative link.
James’s (1890a) law of habit was central to the basic mechanism proposed by
neuroscientist Donald Hebb (1949) for the development of cell assemblies. Hebb
provided a famous modern statement of James’ law of habit:
When an axon of cell A is near enough to excite a cell B and repeatedly or persistently
takes part in firing it, some growth process or metabolic change takes place
in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.
(Hebb, 1949, p. 62)
This makes explicit the modern connectionist idea that learning is modifying the
strength of connections between processors. Hebb’s theory inspired the earliest
computer simulations of memory systems akin to the one proposed by James
(Milner, 1957; Rochester et al., 1956). These simulations revealed a critical role for
inhibition that led Hebb (1959) to revise his theory. Modern neuroscience has discovered
a phenomenon called long-term potentiation that is often cited as a biologically
plausible instantiation of Hebb’s theory (Brown, 1990; Gerstner & Kistler, 2002;
Martinez & Derrick, 1996; van Hemmen & Senn, 2002).
The journey from James through Hebb to the first simulations of memory
(Milner, 1957; Rochester et al., 1956) produced a modern associative memory system
called the standard pattern associator (McClelland, 1986). The standard pattern
associator, which is structurally identical to Figure 4-1, is a memory capable of
learning associations between pairs of input patterns (Steinbuch, 1961; Taylor, 1956)
or learning to associate an input pattern with a categorizing response (Rosenblatt,
1962; Selfridge, 1956; Widrow & Hoff, 1960).
The standard pattern associator is empiricist in the sense that its knowledge
is acquired by experience. Usually the memory begins as a blank slate: all of the
connections between processors start with weights equal to zero. During a learning
Elements of Connectionist Cognitive Science 137