Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

276 11. Rotation and Interpretation of Principal Componentstwo rotated components are 0.21 and 0.35. The rotated components correspondingto à m are uncorrelated, but the angles between the vector ofloadings for the displayed component and those of the other two rotatedcomponents are 61 ◦ and 48 ◦ , respectively, rather than 90 ◦ as required fororthogonality. For the normalization based on Ãm the respective correlationsbetween the plotted component and the other rotated componentsare 0.30 and 0.25, with respective corresponding angles between vectors ofloadings equal to 61 ◦ and 76 ◦ .Artistic Qualities of PaintersIn the SST example, there are quite substantial differences in the rotatedloadings, depending on which normalization constraint is used. Jolliffe(1989) suggests a strategy for rotation that avoids this problem, and alsoalleviates two of the other three drawbacks of rotation noted above. Thisstrategy is to move away from necessarily rotating the first few PCs, butinstead to rotate subsets of components with similar eigenvalues. The effectof different normalizations on rotated loadings depends on the relativelengths of the vectors of loadings within the set of loadings that are rotated.If the eigenvalues are similar for all PCs to be rotated, any normalizationin which lengths of loading vectors are functions of eigenvalues is similar toa normalization in which lengths are constant. The three constraints abovelspecify lengths of 1, k n−1n−1,andl k, and it therefore matters little which isused when eigenvalues in the rotated set are similar.With similar eigenvalues, there is also no dominant PC or PCs withinthe set being rotated, so the second drawback of rotation is removed. Thearbitary choice of m also disappears, although it is replaced by another arbitrarychoice of which sets of eigenvalues are sufficiently similar to justifyrotation. However, there is a clearer view here of what is required (closeeigenvalues) than in choosing m. The latter is multifaceted, depending onwhat the m retained components are meant to achieve, as evidenced bythe variety of possible rules described in Section 6.1. If approximate multivariatenormality can be assumed, the choice of which subsets to rotatebecomes less arbitrary, as tests are available for hypotheses averring thatblocks of consecutive population eigenvalues are equal (Flury and Riedwyl,1988, Section 10.7). These authors argue that when such hypotheses cannotbe rejected, it is dangerous to interpret individual eigenvectors—only thesubspace defined by the block of eigenvectors is well-defined, not individualeigenvectors. A possible corollary of this argument is that PCs correspondingto such subspaces should always be rotated in order to interpret thesubspace as simply as possible.Jolliffe (1989) gives three examples of rotation for PCs with similar eigenvalues.One, which we summarize here, is the four-variable artistic qualitiesdata set described in Section 5.1. In this example the four eigenvalues are2.27, 1.04, 0.40 and 0.29. The first two of these are well-separated, and