Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

42 3. Properties of Sample Principal ComponentsBecause variable PLATE has a variance 100 times larger than any othervariable, the first PC accounts for over 98 percent of the total variation.Thus the first six components for the covariance matrix tell us almost nothingapart from the order of sizes of variances of the original variables. Bycontrast, the first few PCs for the correlation matrix show that certainnon-trivial linear functions of the (standardized) original variables accountfor substantial, though not enormous, proportions of the total variation inthe standardized variables. In particular, a weighted contrast between thefirst four and the last four variables is the linear function with the largestvariance.This example illustrates the dangers in using a covariance matrix to findPCs when the variables have widely differing variances; the first few PCswill usually contain little information apart from the relative sizes of variances,information which is available without a PCA. There are, however,circumstances in which it has been argued that using the covariance matrixhas some advantages; see, for example, Naik and Khattree (1996), althoughthese authors transform their data (track record times for Olympic eventsare transformed to speeds) in order to avoid highly disparate variances.Apart from the fact already mentioned in Section 2.3, namely, that itis more difficult to base statistical inference regarding PCs on correlationmatrices, one other disadvantage of correlation matrix PCs is that theygive coefficients for standardized variables and are therefore less easy to interpretdirectly. To interpret the PCs in terms of the original variables eachcoefficient must be divided by the standard deviation of the correspondingvariable. An example which illustrates this is given in the next section.It must not be forgotten, however, that correlation matrix PCs, when reexpressedin terms of the original variables, are still linear functions of xthat maximize variance with respect to the standardized variables and notwith respect to the original variables.An alternative to finding PCs for either covariance or correlation matricesis to calculate the eigenvectors of ˜X ′ ˜X rather than X ′ X, that is, measurevariables about zero, rather than about their sample means, when computing‘covariances’ and ‘correlations.’ This idea was noted by Reyment andJöreskog (1993, Section 5.4) and will be discussed further in Section 14.2.3.‘Principal component analysis’ based on measures of association of thisform, but for observations rather than variables, has been found useful forcertain types of geological data (Reyment and Jöreskog, 1993). Anothervariant, in a way the opposite of that just mentioned, has been used byBuckland and Anderson (1985), among others. Their idea, which is alsodiscussed further in Section 14.2.3, and which again is appropriate for aparticular type of data, is to ‘correct for the mean’ in both the rows andcolumns of ˜X. Further possibilities, such as the use of weights or metrics,are described in Section 14.2.