Mind, Body, World- Foundations of Cognitive Science, 2013a

The problem with reverse engineering a black box is this: if there are potentially
many different algorithms that can produce the same input-output mapping, then
mere observations of input-output behaviour will not by themselves indicate which
particular algorithm is used in the device’s design. However, reverse engineering a
black box is not impossible. In addition to the behaviours that it was designed to
produce, the black box will also produce artifacts. Artifacts can provide great deal
of information about internal and unobservable algorithms.
Imagine that we are faced with reverse engineering an arithmetic calculator
that is also a black box. Some of the artifacts of this calculator provide relative
complexity evidence (Pylyshyn, 1984). To collect such evidence, one could conduct
an experiment in which the problems presented to the calculator were systematically
varied (e.g., by using different numbers) and measurements were made of the
amount of time taken for the correct answer to be produced. To analyze this relative
complexity evidence, one would explore the relationship between characteristics of
problems and the time required to solve them.
For instance, suppose that one observed a linear increase in the time taken to
solve the problems 9 × 1, 9 × 2, 9 × 3, et cetera. This could indicate that the device
was performing multiplication by doing repeated addition (9, 9 + 9, 9 + 9 + 9, and so
on) and that every “+ 9” operation required an additional constant amount of time
to be carried out. Psychologists have used relative complexity evidence to investigate
cognitive algorithms since Franciscus Donders invented his subtractive method in
1869 (Posner, 1978).
Artifacts can also provide intermediate state evidence (Pylyshyn, 1984).
Intermediate state evidence is based upon the assumption that an input-output
mapping is not computed directly, but instead requires a number of different stages
of processing, with each stage representing an intermediate result in a different way.
To collect intermediate state evidence, one attempts to determine the number and
nature of these intermediate results.
For some calculating devices, intermediate state evidence can easily be collected.
For instance, the intermediate states of the Turing machine›s tape, the
abacus’ beads or the difference engine’s gears are in full view. For other devices,
though, the intermediate states are hidden from direct observation. In this case,
clever techniques must be developed to measure internal states as the device is
presented with different inputs. One might measure changes in electrical activity
in different components of an electronic calculator as it worked, in an attempt to
acquire intermediate state evidence.
Artifacts also provide error evidence (Pylyshyn, 1984), which may also help to
explore intermediate states. When extra demands are placed on a system’s resources,
it may not function as designed, and its internal workings are likely to become more
evident (Simon, 1969). This is not just because the overtaxed system makes errors
44 Chapter 2