Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

8 1. IntroductionAfter giving the derivation, Hotelling goes on to show how to find thecomponents using the power method (see Appendix A1). He also discussesa different geometric interpretation from that given by Pearson, in terms ofellipsoids of constant probability for multivariate normal distributions (seeSection 2.2). A fairly large proportion of his paper, especially the secondpart, is, however, taken up with material that is not concerned with PCAin its usual form, but rather with factor analysis (see Chapter 7).A further paper by Hotelling (1936) gave an accelerated version of thepower method for finding PCs; in the same year, Girshick (1936) providedsome alternative derivations of PCs, and introduced the idea that samplePCs were maximum likelihood estimates of underlying population PCs.Girshick (1939) investigated the asymptotic sampling distributions of thecoefficients and variances of PCs, but there appears to have been only asmall amount of work on the development of different applications of PCAduring the 25 years immediately following publication of Hotelling’s paper.Since then, however, an explosion of new applications and further theoreticaldevelopments has occurred. This expansion reflects the general growthof the statistical literature, but as PCA requires considerable computingpower, the expansion of its use coincided with the widespread introductionof electronic computers. Despite Pearson’s optimistic comments, it is not reallyfeasible to do PCA by hand, unless p is about four or less. But it is preciselyfor larger values of p that PCA is most useful, so that the full potentialof the technique could not be exploited until after the advent of computers.Before ending this section, four papers will be mentioned; these appearedtowards the beginning of the expansion of interest in PCA and have becomeimportant references within the subject. The first of these, by Anderson(1963), is the most theoretical of the four. It discussed the asymptoticsampling distributions of the coefficients and variances of the sample PCs,building on the earlier work by Girshick (1939), and has been frequentlycited in subsequent theoretical developments.Rao’s (1964) paper is remarkable for the large number of new ideas concerninguses, interpretations and extensions of PCA that it introduced, andwhich will be cited at numerous points in the book.Gower (1966) discussed links between PCA and various other statisticaltechniques, and also provided a number of important geometric insights.Finally, Jeffers (1967) gave an impetus to the really practical side of thesubject by discussing two case studies in which the uses of PCA go beyondthat of a simple dimension-reducing tool.To this list of important papers the book by Preisendorfer and Mobley(1988) should be added. Although it is relatively unknown outside thedisciplines of meteorology and oceanography and is not an easy read, itrivals Rao (1964) in its range of novel ideas relating to PCA, some ofwhich have yet to be fully explored. The bulk of the book was written byPreisendorfer over a number of years, but following his untimely death themanuscript was edited and brought to publication by Mobley.