Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

134 6. Choosing a Subset of Principal Components or VariablesTable 6.2. First six eigenvalues for the covariance matrix, blood chemistry data.Component number 1 2 3 4 5 6Eigenvalue, l k 1704.68 15.07 6.98 2.64 0.13 0.07l k /¯l ∑ 7.88 0.07 0.03 0.01 0.0006 0.0003mk=1t m = 100l k∑ p l 98.6 99.4 99.8 99.99 99.995 99.9994k=1 kl k−1 − l k 1689.61 8.09 4.34 2.51 0.06nearly 80% of the total variation, but it takes six PCs to account for 90%.A cut-off at l ∗ =0.7 for the second criterion retains four PCs, but the nexteigenvalue is not very much smaller, so perhaps five should be retained. Inthe scree graph the slope actually increases between k = 3 and 4, but thenfalls sharply and levels off, suggesting that perhaps only four PCs shouldbe retained. The LEV diagram (not shown) is of little help here; it has noclear indication of constant slope after any value of k, and in fact has itssteepest slope between k =7and8.Using Cattell’s (1966) formulation, there is no strong straight-line behaviourafter any particular point, although perhaps a cut-off at k =4ismost appropriate. Cattell suggests that the first point on the straight line(that is, the ‘elbow’ point) should be retained. However, if we consider thescree graph in the same light as the test of Section 6.1.4, then all eigenvaluesafter, and including, the elbow are deemed roughly equal and so allcorresponding PCs should be deleted. This would lead to the retention ofonly three PCs in the present case.Turning to Table 6.2, which gives information for the covariance matrix,corresponding to that presented for the correlation matrix in Table 6.1, thethree ad hoc measures all conclusively suggest that one PC is sufficient. Itis undoubtedly true that choosing m = 1 accounts for the vast majorityof the variation in x, but this conclusion is not particularly informativeas it merely reflects that one of the original variables accounts for nearlyall the variation in x. The PCs for the covariance matrix in this examplewere discussed in Section 3.3, and it can be argued that it is the use ofthe covariance matrix, rather than the rules of Sections 6.1.1–6.1.3, that isinappropriate for these data.6.2.2 Gas Chromatography DataThese data, which were originally presented by McReynolds (1970), andwhich have been analysed by Wold (1978) and by Eastment and Krzanowski(1982), are concerned with gas chromatography retention indices. Afterremoval of a number of apparent outliers and an observation with a missingvalue, there remain 212 (Eastment and Krzanowski) or 213 (Wold) measurementson ten variables. Wold (1978) claims that his method indicates