Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

2.3. Principal Components Using a Correlation Matrix 212.3 Principal Components Using a CorrelationMatrixThe derivation and properties of PCs considered above are based on theeigenvectors and eigenvalues of the covariance matrix. In practice, as willbe seen in much of the remainder of this text, it is more common to defineprincipal components asz = A ′ x ∗ , (2.3.1)where A now has columns consisting of the eigenvectors of the correlationmatrix, and x ∗ consists of standardized variables. The goal in adoptingsuch an approach is to find the principal components of a standardized,j=1, 2,...,p, x j isthe jth element of x, andσ jj is the variance of x j . Then the covariancematrix for x ∗ is the correlation matrix of x, and the PCs of x ∗ are givenby (2.3.1).A third possibility, instead of using covariance or correlation matrices,is to use covariances of x j /w j , where the weights w j are chosen to reflectsome a priori idea of the relative importance of the variables. The specialversion x ∗ of x, where x ∗ has jth element x j /σ 1/2jjleads to x ∗ , and to PCs based on the correlation matrix,but various authors have argued that the choice of w j = σ 1/2jj is somewhatarbitrary, and that different values of w j might be better in some applications(see Section 14.2.1). In practice, however, it is relatively unusual thata uniquely appropriate set of w j suggests itself.All the properties of the previous two sections are still valid for correlationmatrices, or indeed for covariances based on other sets of weights,except that we are now considering PCs of x ∗ (or some other transformationof x), instead of x.It might seem that the PCs for a correlation matrix could be obtainedfairly easily from those for the corresponding covariance matrix, since x ∗is related to x by a very simple transformation. However, this is not thecase; the eigenvalues and eigenvectors of the correlation matrix have nosimple relationship with those of the corresponding covariance matrix. Inparticular, if the PCs found from the correlation matrix are expressed interms of x by transforming back from x ∗ to x, then these PCs are notthe same as the PCs found from Σ, except in very special circumstances(Chatfield and Collins, 1989, Section 4.4). One way of explaining this isthat PCs are invariant under orthogonal transformations of x but not, ingeneral, under other transformations (von Storch and Zwiers, 1999, Section13.1.10). The transformation from x to x ∗ is not orthogonal. The PCsfor correlation and covariance matrices do not, therefore, give equivalentinformation, nor can they be derived directly from each other. We nowdiscuss the relative merits of the two types of PC.case w j = σ 1/2jj