Mind, Body, World- Foundations of Cognitive Science, 2013a

instance, can any classical theory be translated into a network? This is one type of
algorithmic-level issue that requires a great deal of additional research. As well, the
translation works both ways: perhaps artificial neural networks provide a biologically
plausible lingua franca for classical architectures!
4.13 Of Coarse Codes
The notion of representation in classical cognitive science is tightly linked to the
structure/process distinction that is itself inspired by the digital computer. An
explicit set of rules is proposed to operate on a set of symbols that permits its components
to be identified, digitally, as tokens that belong to particular symbol types.
In contrast, artificial neural networks dispense (at first glance) with the sharp
distinction between structure and process that characterizes classical cognitive science.
Instead, networks themselves take the form of dynamic symbols that represent
information at the same time as they transform it. The dynamic, distributed nature
of artificial neural networks appears to make them more likely to be explained using
statistical mechanics than using propositional logic.
One of the putative advantages of connectionist cognitive science is that it can
inspire alternative notions of representation. The blurring of the structure/process
distinction, the seemingly amorphous nature of the internal structure that characterizes
many multilayer networks, leads to one such proposal, called coarse coding.
A coarse code is one in which an individual unit is very broadly tuned, sensitive
to either a wide range of features or at least to a wide range of values for an individual
feature (Churchland & Sejnowski, 1992; Hinton, McClelland, & Rumelhart, 1986).
In other words, individual processors are themselves very inaccurate devices for
measuring or detecting a feature. The accurate representation of a feature can
become possible, though, by pooling or combining the responses of many such inaccurate
detectors, particularly if their perspectives are slightly different (e.g., if they
are sensitive to different ranges of features, or if they detect features from different
input locations).
A familiar example of coarse coding is provided by the nineteenth trichromatic
theory of colour perception (Helmholtz, 1968; Wasserman, 1978). According to this
theory, colour perception is mediated by three types of retinal cone receptors. One is
maximally sensitive to short (blue) wavelengths of light, another is maximally sensitive
to medium (green) wavelengths, and the third is maximally sensitive to long
(red) wavelengths. Thus none of these types of receptors are capable of representing,
by themselves, the rich rainbow of perceptible hues.
However, these receptors are broadly tuned and have overlapping sensitivities. As
a result, most light will activate all three channels simultaneously, but to different
degrees. Actual colored light does not produce sensations of absolutely pure
184 Chapter 4