Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

7Principal Component Analysis andFactor AnalysisPrincipal component analysis has often been dealt with in textbooks as aspecial case of factor analysis, and this practice is continued by some widelyused computer packages, which treat PCA as one option in a program forfactor analysis. This view is misguided since PCA and factor analysis, asusually defined, are really quite distinct techniques. The confusion mayhave arisen, in part, because of Hotelling’s (1933) original paper, in whichprincipal components were introduced in the context of providing a smallnumber of ‘more fundamental’ variables that determine the values of thep original variables. This is very much in the spirit of the factor modelintroduced in Section 7.1, although Girschick (1936) indicates that therewere soon criticisms of Hotelling’s PCs as being inappropriate for factoranalysis. Further confusion results from the fact that practitioners of ‘factoranalysis’ do not always have the same definition of the technique (seeJackson, 1991, Section 17.1). In particular some authors, for example Reymentand Jöreskog (1993), Benzécri (1992, Section 4.3) use the term toembrace a wide spectrum of multivariate methods. The definition adoptedin this chapter is, however, fairly standard.Both PCA and factor analysis aim to reduce the dimensionality of aset of data, but the approaches taken to do so are different for the twotechniques. Principal component analysis has been extensively used as partof factor analysis, but this involves ‘bending the rules’ that govern factoranalysis and there is much confusion in the literature over the similaritiesand differences between the techniques. This chapter attempts to clarifythe issues involved, and starts in Section 7.1 with a definition of the basicmodel for factor analysis. Section 7.2 then discusses how a factor model