Mind, Body, World- Foundations of Cognitive Science, 2013a

connectionist cognitive science develops models that are subsymbolic (Smolensky,
1988) and which can be described as statistical pattern recognizers. Networks use
representations (Dawson, 2004; Horgan & Tienson, 1996), but these representations
do not have the syntactic structure of those found in classical models (Waskan &
Bechtel, 1997). Let us take a moment to describe in a bit more detail the basic properties
of artificial neural networks.
An artificial neural network is a computer simulation of a “brain-like” system
of interconnected processing units (see Figures 4-1 and 4-5 later in this chapter). In
general, such a network can be viewed as a multiple-layer system that generates a
desired response to an input stimulus. That is, like the devices described by cybernetics
(Ashby, 1956, 1960), an artificial neural network is a machine that computes a
mapping between inputs and outputs.
A network’s stimulus or input pattern is provided by the environment and is
encoded as a pattern of activity (i.e., a vector of numbers) in a set of input units.
The response of the system, its output pattern, is represented as a pattern of activity
in the network›s output units. In modern connectionism—sometimes called
New Connectionism—there will be one or more intervening layers of processors
in the network, called hidden units. Hidden units detect higher-order features in
the input pattern, allowing the network to make a correct or appropriate response.
The behaviour of a processor in an artificial neural network, which is analogous
to a neuron, can be characterized as follows. First, the processor computes the total
signal (its net input) being sent to it by other processors in the network. Second, the
unit uses an activation function to convert its net input into internal activity (usually
a continuous number between 0 and 1) on the basis of this computed signal.
Third, the unit converts its internal activity into an output signal, and sends this
signal on to other processors. A network uses parallel processing because many, if
not all, of its units will perform their operations simultaneously.
The signal sent by one processor to another is a number that is transmitted
through a weighted connection, which is analogous to a synapse. The connection
serves as a communication channel that amplifies or attenuates signals being sent
through it, because these signals are multiplied by the weight associated with the
connection. The weight is a number that defines the nature and strength of the connection.
For example, inhibitory connections have negative weights, and excitatory
connections have positive weights. Strong connections have strong weights (i.e., the
absolute value of the weight is large), while weak connections have near-zero weights.
The pattern of connectivity in a PDP network (i.e., the network’s entire set of
connection weights) defines how signals flow between the processors. As a result,
a network’s connection weights are analogous to a program in a conventional computer
(Smolensky, 1988). However, a network’s “program” is not of the same type
that defines a classical model. A network’s program does not reflect the classical
Elements of Connectionist Cognitive Science 129