Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

132 6. Choosing a Subset of Principal Components or Variablessecond and third largest eigenvalues of the fixed matrix Z, so he arguesthat m = 2 should be chosen. This implies a movement away from theobjective ‘correct’ choice given by the model, back towards what seems tobe the inevitable subjectivity of the area.The simulations are replicated 100 times for each of the two noise levels,and give results which are consistent with other studies. Kaiser’s modifiedrule with a threshold at 2, the broken stick rule, Velicer’s test, and crossvalidationrules that stop after the first fall below the threshold—all retainrelatively few components. Conversely, Bartlett’s test, cumulative variancewith a cut-off of 90%, ˆfq and the approximate jackknife retain greaternumbers of PCs. The approximate jackknife displays the strange behaviourof retaining more PCs for larger than for smaller noise levels. If we considerm = 8 to be ‘correct’ for both noise levels, all rules behave poorly for thehigh noise level. For the low noise level, ˆfq and Bartlett’s tests do best.If m = 2 is deemed correct for the high noise level, the best proceduresare Kaiser’s modified rule with threshold 2, the scree graph, and all fourvarieties of cross-validation. Even within this restricted study no rule isconsistently good.Bartkowiak (1991) gives an empirical comparison for some meteorologicaldata of: subjective rules based on cumulative variance and on the screeand LEV diagrams; the rule based on eigenvalues greater than 1 or 0.7; thebroken stick rule; Velicer’s criterion. Most of the rules lead to similar decisions,except for the broken stick rule, which retains too few components,and the LEV diagram, which is impossible to interpret unambiguously.The conclusion for the broken stick rule is the opposite of that in Jackson’s(1993) study.Throughout our discussion of rules for choosing m we have emphasizedthe descriptive rôle of PCA and contrasted it with the model-basedapproach of factor analysis. It is usually the case that the number of componentsneeded to achieve the objectives of PCA is greater than the numberof factors in a factor analysis of the same data. However, this need notbe the case when a model-based approach is adopted for PCA (see Sections3.9, 6.1.5). As Heo and Gabriel (2001) note in the context of biplots(see Section 5.3), the fit of the first few PCs to an underlying populationpattern (model) may be much better than their fit to a sample. This impliesthat a smaller value of m may be appropriate for model-based PCAthan for descriptive purposes. In other circumstances, too, fewer PCs maybe sufficient for the objectives of the analysis. For example, in atmosphericscience, where p can be very large, interest may be restricted only to thefirst few dominant and physically interpretable patterns of variation, eventhough their number is fewer than that associated with most PCA-basedrules. Conversely, sometimes very dominant PCs are predictable and henceof less interest than the next few. In such cases more PCs will be retainedthan indicated by most rules. The main message is that different objectivesfor a PCA lead to different requirements concerning how many PCs