Mind, Body, World- Foundations of Cognitive Science, 2013a

depolarization of the membrane of the neuron’s axon, called an action potential,
which is a signal of constant intensity that travels along the axon to eventually stimulate
some other neuron.
A crucial property of the action potential is that it is an all-or-none phenomenon,
representing a nonlinear transformation of the summed graded potentials.
The neuron converts continuously varying inputs into a response that is either on
(action potential generated) or off (action potential not generated). This has been
called the all-or-none law (Levitan & Kaczmarek, 1991, p. 43): “The all-or-none law
guarantees that once an action potential is generated it is always full size, minimizing
the possibility that information will be lost along the way.” The all-or-none
output of neurons is a nonlinear transformation of summed, continuously varying
input, and it is the reason that the brain can be described as digital in nature (von
Neumann, 1958).
The all-or-none behaviour of a neuron makes it logically equivalent to the
relays or switches that were discussed in Chapter 2. This logical interpretation was
exploited in an early mathematical account of the neural information processing
(McCulloch & Pitts, 1943). McCulloch and Pitts used the all-or-none law to justify
describing neurons very abstractly as devices that made true or false logical assertions
about input information:
The all-or-none law of nervous activity is sufficient to insure that the activity of any
neuron may be represented as a proposition. Physiological relations existing among
nervous activities correspond, of course, to relations among the propositions; and
the utility of the representation depends upon the identity of these relations with
those of the logical propositions. To each reaction of any neuron there is a corresponding
assertion of a simple proposition. (McCulloch & Pitts, 1943, p. 117)
McCulloch and Pitts (1943) invented a connectionist processor, now known as the
McCulloch-Pitts neuron (Quinlan, 1991), that used the all-or-none law. Like the
output units in the standard pattern associator (Figure 4-1), a McCulloch-Pitts
neuron first computes its net input by summing all of its incoming signals. However,
it then uses a nonlinear activation function to transform net input into internal
activity. The activation function used by McCulloch and Pitts was the Heaviside
step function, named after nineteenth-century electrical engineer Oliver Heaviside.
This function compares the net input to a threshold. If the net input is less than the
threshold, the unit’s activity is equal to 0. Otherwise, the unit’s activity is equal to
1. (In other artificial neural networks [Rosenblatt, 1958, 1962], below-threshold net
inputs produced activity of –1.)
The output units in the standard pattern associator (Figure 4-1) can be
described as using the linear identity function to convert net input into activity,
because output unit activity is equal to net input. If one replaced the identity function
with the Heaviside step function in the standard pattern associator, it would
140 Chapter 4