Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

11.1. Rotation of Principal Components 271It is certainly possible to simplify interpretion of PCs by using rotation.For example, Table 7.2 gives two versions of rotated loadings for the firstfour PCs in a study of scores on 10 intelligence tests for 150 children fromthe Isle of Wight. The two versions correspond to varimax rotation andto an oblique rotation criterion, direct quartimin. The unrotated loadingsare given in Table 7.1. The type of simplicity favoured by almost all rotationcriteria attempts to drive loadings towards zero or towards theirmaximum possible absolute value, which with most scalings of the loadingsis 1. The idea is that it should then be clear which variables are ‘important’in a (rotated) component, namely, those with large absolute valuesfor their loadings, and those which are not important (loadings near zero).Intermediate-value loadings, which are difficult to interpret, are avoidedas much as possible by the criteria. Comparing Tables 7.1 and 7.2, it isapparent that this type of simplicity has been achieved by rotation.There are other types of simplicity. For example, the first unrotatedcomponent in Table 7.1 has all its loadings of similar magnitude and thesame sign. The component is thus simple to interpret, as an average ofscores on all ten tests. This type of simplicity is shunned by most rotationcriteria, and it is difficult to devise a criterion which takes into account morethan one type of simplicity, though Richman (1986) attempts to broadenthe definition of simple structure by graphical means.In the context of PCA, rotation has a number of drawbacks:• A choice has to made from a large number of possible rotation criteria.Cattell (1978) and Richman (1986), respectively, give non-exhaustivelists of 11 and 19 such criteria. Frequently, the choice is made arbitrarily,for example by using the default criterion in a computer package(often varimax). Fortunately, as noted already, different choices of criteria,at least within orthogonal rotation, often make little differenceto the results.• PCA successively maximizes variance accounted for. When rotationis done, the total variance within the rotated m-dimensional subspaceremains unchanged; it is still the maximum that can be achieved, butit is redistributed amongst the rotated components more evenly thanbefore rotation. This means that information about the nature ofany really dominant components may be lost, unless they are already‘simple’ in the sense defined by the chosen rotation criterion.• The choice of m can have a large effect on the results after rotation.This is illustrated in Tables 7.4 and 7.2 when moving from m =3to m = 4 for the children’s intelligence example. Interpreting the‘most important dimensions’ for a data set is clearly harder if those‘dimensions’ appear, disappear, and possibly reappear, as m changes.• The choice of normalization constraint used for the columns in thematrix A m changes the properties of the rotated loadings. From the