Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

9.2. Cluster Analysis 219county cluster in the top right of the plot splits into three groups containing13, 10 and 4 counties, with some overlap between them.This example is typical of many in which cluster analysis is used fordissection. Examples like that of Jeffers’ (1967) aphids, where a very clearcutand previously unknown cluster structure is uncovered, are relativelyunusual, although another illustration is given by Blackith and Reyment(1971, p. 155). In their example, a plot of the observations with respect tothe second and third (out of seven) PCs shows a very clear separation intotwo groups. It is probable that in many circumstances ‘projection-pursuit’methods, which are discussed next, will provide a better two-dimensionalspace in which to view the results of a cluster analysis than that defined bythe first two PCs. However, if dissection rather than discovery of a clear-cutcluster structure is the objective of a cluster analysis, then there is likelyto be little improvement over a plot with respect to the first two PCs.9.2.2 Projection PursuitAs mentioned earlier in this chapter, it may be possible to find lowdimensionalrepresentations of a data set that are better than the first fewPCs at displaying ‘structure’ in the data. One approach to doing this is todefine structure as ‘interesting’ and then construct an index of ‘interestingness,’which is successively maximized. This is the idea behind projectionpursuit, with different indices leading to different displays. If ‘interesting’is defined as ‘large variance,’ it is seen that PCA is a special case of projectionpursuit. However, the types of structure of interest are often clustersor outliers, and there is no guarantee that the high-variance PCs will findsuch features. The term ‘projection pursuit’ dates back to Friedman andTukey (1974), and a great deal of work was done in the early 1980s. Thisis described at length in three key papers: Friedman (1987), Huber (1985),and Jones and Sibson (1987). The last two both include extensive discussion,in addition to the paper itself. Some techniques are good at findingclusters, whereas others are better at detecting outliers.Most projection pursuit techniques start from the premise that the leastinteresting structure is multivariate normality, so that deviations fromnormality form the basis of many indices. There are measures based onskewness and kurtosis, on entropy, on looking for deviations from uniformityin transformed data, and on finding ‘holes’ in the data. More recently,Foster (1998) suggested looking for directions of high density, after ‘sphering’the data to remove linear structure. Sphering operates by transformingthe variables x to z = S − 1 2 (x−¯x), which is equivalent to converting to PCs,which are then standardized to have zero mean and unit variance. Friedman(1987) also advocates sphering as a first step in his version of projectionpursuit. After identifying the high-density directions for the sphered data,Foster (1998) uses the inverse transformation to discover the nature of theinteresting structures in terms of the original variables.