Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

7.5. Concluding Remarks 165To illustrate what happens when different numbers of factors are retained,Table 7.4 gives factor loadings for three factors using varimaxrotation. The loadings for direct quartimin (not shown) are again very similar.Before rotation, changing the number of PCs simply adds or deletesPCs, leaving the remaining PCs unchanged. After rotation, however, deletionor addition of factors will usually change all of the factor loadings.In the present example, deletion of the fourth unrotated factor leaves thefirst rotated factor almost unchanged, except for a modest increase in theloading for variable 4. Factor 2 here is also similar to factor 2 in the fourfactoranalysis, although the resemblance is somewhat less strong than forfactor 1. In particular, variable 6 now has the largest loading in factor 2,whereas previously it had only the fourth largest loading. The third factorin the three-factor solution is in no way similar to factor 3 in the four-factoranalysis. In fact, it is quite similar to the original factor 4, and the originalfactor 3 has disappeared, with its highest loadings on variables 4 and 6partially ‘transferred’ to factors 1 and 2, respectively.The behaviour displayed in this example, when a factor is deleted, isnot untypical of what happens in factor analysis generally, although the‘mixing-up’ and ‘rearrangement’ of factors can be much more extreme thanin the present case.7.5 Concluding RemarksFactor analysis is a large subject, and this chapter has concentrated onaspects that are most relevant to PCA. The interested reader is referred toone of the many books on the subject such as Cattell (1978), Lawley andMaxwell (1971), Lewis-Beck (1994) or Rummell (1970) for further details.Factor analysis is one member of the class of latent variable models (seeBartholomew and Knott (1999)) which have been the subject of muchrecent research. Mixture modelling, discussed in Section 9.2.3, is anotherof the many varieties of latent variable models.It should be clear from the discussion of this chapter that it does notreally make sense to ask whether PCA is ‘better than’ factor analysis orvice versa, because they are not direct competitors. If a model such as(7.1.2) seems a reasonable assumption for a data set, then factor analysis,rather than PCA, is appropriate. If no such model can be assumed, thenfactor analysis should not really be used.Despite their different formulations and objectives, it can be informativeto look at the results of both techniques on the same data set. Eachtechnique gives different insights into the data structure, with PCA concentratingon explaining the diagonal elements, and factor analysis theoff-diagonal elements, of the covariance matrix, and both may be useful.Furthermore, one of the main ideas of factor analysis, that of rotation, can