Mind, Body, World- Foundations of Cognitive Science, 2013a

anding when a hidden unit’s activities were plotted using a jittered density plot
(Berkeley et al., 1995). A jittered density plot is a two-dimensional scatterplot of
points; one such plot can be created for each hidden unit in a network. Each plotted
point represents one of the patterns presented to the hidden unit during wiretapping.
The x-value of the point’s position in the graph is the activity produced in that
hidden unit by the pattern. The y-value of the point’s position in the scatterplot is a
random value that is assigned to reduce overlap between points.
An example of a jittered density plot for a hidden value unit is provided in
Figure 4-15. Note that the points in this plot are organized into distinct bands,
which is consistent with the geometric analysis. This particular unit belongs to a
network of value units trained on a logic problem discussed in slightly more detail
below (Bechtel & Abrahamsen, 1991), and was part of a study that examined some of
the implications of activity banding (Dawson & Piercey, 2001).
0.0 0.2 0.4 0.6 0.8 1.0
Hidden Unit Activity
Figure 4-15. An example of banding in a jittered density plot of a hidden value
unit in a network that was trained on a logic problem.
Bands in jittered density plots of hidden value units can be used to reveal the kinds
of features that are being detected by these units. For instance, Berkeley et al. (1995)
reported that all of the patterns that fell into the same band on a single jittered
density plot in the networks did so because they shared certain local properties or
features, which are called definite features.
There are two types of definite features. The first is called a definite unary
feature. When a definite unary feature exists, it means that a single feature has the
same value for every pattern in the band. The second is called a definite binary feature.
With this kind of definite feature, an individual feature is not constant within
Elements of Connectionist Cognitive Science 175