Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

26 2. Properties of Population Principal ComponentsThe criterionp∑j=1R 2 j:qis maximized when y 1 ,y 2 ,...,y q are the first q correlation matrix PCs.The maximized value of the criterion is equal to the sum of the q largesteigenvalues of the correlation matrix.Because the principal components are uncorrelated, the criterion inProperty A6 reduces top∑q∑rjk2j=1 k=1where rjk2 is the squared correlation between the jth variable and thekth PC. The criterion will be maximized by any matrix B that givesy spanning the same q-dimensional space as the first q PCs. However,the correlation matrix PCs are special, in that they successivelymaximize the criterion for q = 1, 2,...,p. As noted following PropertyA5, this result was given by Hotelling (1933) alongside his originalderivation of PCA, but it has subsequently been largely ignored. It isclosely related to Property A5. Meredith and Millsap (1985) derivedProperty A6 independently and noted that optimizing the multiple correlationcriterion gives a scale invariant method (as does Property A5;Cadima, 2000). One implication of this scale invariance is that it givesadded importance to correlation matrix PCA. The latter is not simplya variance-maximizing technique for standardized variables; its derivedvariables are also the result of optimizing a criterion which is scaleinvariant, and hence is relevant whether or not the variables are standardized.Cadima (2000) discusses Property A6 in greater detail andargues that optimization of its multiple correlation criterion is actuallya new technique, which happens to give the same results as correlationmatrix PCA, but is broader in its scope. He suggests that thederived variables be called Most Correlated Components. Lookedatfromanother viewpoint, this broader relevance of correlation matrix PCAgives another reason to prefer it over covariance matrix PCA in mostcircumstances.To conclude this discussion, we note that Property A6 can be easilymodified to give a new property for covariance matrix PCA. The first qcovariance marix PCs maximize, amongst all orthonormal linear transformationsof x, thesumofsquaredcovariances between x 1 ,x 2 ,...,x p andthe derived variables y 1 ,y 2 ,...,y q . Covariances, unlike correlations, are notscale invariant, and hence neither is covariance matrix PCA.