Mind, Body, World- Foundations of Cognitive Science, 2013a

easons” as would the decision tree. This is why Dawson et al. hoped that this classical
theory would literally be translated into the network.
Apart from the output unit behaviour, how could one support the claim that
a classical theory had been translated into a connectionist network? Dawson et
al. (2000) interpreted the internal structure of the network in an attempt to see
whether such a network analysis would reveal an internal representation of the
classical algorithm. If this were the case, then standard training practices would
have succeeded in translating the classical algorithm into a PDP network.
One method that Dawson et al. (2000) used to interpret the trained network
was a multivariate analysis of the network’s hidden unit space. They represented
each mushroom as the vector of five hidden unit activation values that it produced
when presented to the network. They then performed a k-means clustering of this
data. The k-means clustering is an iterative procedure that assigns data points to k
different clusters in such a way that each member of a cluster is closer to the centroid
of that cluster than to the centroid of any other cluster to which other data
points have been assigned.
However, whenever cluster analysis is performed, one question that must be
answered is How many clusters should be used?—in other words, what should the
value of k be?. An answer to this question is called a stopping rule. Unfortunately,
no single stopping rule has been agreed upon (Aldenderfer & Blashfield, 1984;
Everitt, 1980). As a result, there exist many different types of methods for determining
k (Milligan & Cooper, 1985).
While no general method exists for determining the optimal number of clusters,
one can take advantage of heuristic information concerning the domain being
clustered in order to come up with a satisfactory stopping rule for this domain.
Dawson et al. (2000) argued that when the hidden unit activities of a trained network
are being clustered, there must be a correct mapping from these activities
to output responses, because one trained network itself has discovered one such
mapping. They used this position to create the following stopping rule: “Extract the
smallest number of clusters such that every hidden unit activity vector assigned to
the same cluster produces the same output response in the network.” They used this
rule to determine that the k-means analysis of the network’s hidden unit activity
patterns required the use of 12 different clusters.
Dawson et al. (2000) then proceeded to examine the mushroom patterns that
belonged to each cluster in order to determine what they had in common. For each
cluster, they determined the set of descriptive features that each mushroom shared.
They realized that each set of shared features they identified could be thought of as
a condition, represented internally by the network as a vector of hidden unit activities,
which results in the network producing a particular action, in particular, the
edible/poisonous judgement represented by the first output unit.
182 Chapter 4