Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

272 11. Rotation and Interpretation of Principal Componentstheory of PCA, it is natural to expect the columns of A m to benormalized so as to have unit length, but it is far more commonin computer packages to rotate a matrix Ãm whose columns havelengths equal to their corresponding eigenvalues, as in the sampleversion of equation (2.3.2). The main reason for this convention isalmost certainly due to the fact that it has been borrowed from factoranalysis. This is discussed further later in this section, but first theproperties associated with the rotated components under these twonormalizations are explored.In PCA the components possess two ‘orthogonality’ properties. FirstA ′ mA m = I m , (11.1.1)where I m is the identity matrix of order m. Hence, the vectors of loadings fordifferent components are orthogonal. Second, if A and Z are, as in previouschapters, the (p×p) matrix of loadings or coefficients and the (n×p) matrixof scores, respectively, for all p PCs, and L 2 , as in Chapter 5, is the diagonalmatrix whose elements are eigenvalues of X ′ X, thenZ ′ Z = A ′ X ′ XA = A ′ AL 2 A ′ A = L 2 . (11.1.2)The second equality in (11.1.2) follows from the algebra below equation(5.3.6), and the last equality is a result of the orthogonality of A. Theimplication of this result is that all the p unrotated components, includingthe first m, are uncorrelated with other.The fact that orthogonality of vectors of loadings and uncorrelatednessof component scores both hold for PCs is because the loadings are given byeigenvectors of the covariance matrix S corresponding to the result for Σin Section 1.1. After rotation, one or both of these properties disappears.Let Z m = XA m be the (n × m) matrix of PC scores for n observations onthe first m PCs, so thatZ R m = XB m = XA m T = Z m T (11.1.3)is the corresponding matrix of rotated PC scores. ConsiderandB ′ mB m = T ′ A ′ mA m T, (11.1.4)Z ′R m Z R m = T ′ Z ′ mZ m T. (11.1.5)With the usual PCA normalization a ′ k a k =1,orA ′ mA m = I m , equation(11.1.4) becomes T ′ T, which equals I m for orthogonal rotation. However,if L 2 m is the (m × m) submatrix of L 2 consisting of its first m rows andcolumns then, from (11.1.2), equation (11.1.5) becomes T ′ L 2 mT, whichisnot diagonal. Hence the rotated components using this normalization haveorthogonal loadings, but are not uncorrelated.Next consider the common alternative normalization, corresponding toequation (2.3.2), in which each column a k of A m is multiplied by the square