Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

6.1. How Many Principal Components? 117which the scree graph defines a more-or-less straight line, not necessarilyhorizontal. The first point on the straight line is then taken to be the lastfactor/component to be retained. If there are two or more straight linesformed by the lower eigenvalues, then the cut-off is taken at the upper (lefthand)end of the left-most straight line. Cattell (1966) discusses at somelength whether the left-most point on the straight line should correspondto the first excluded factor or the last factor to be retained. He concludesthat it is preferable to include this factor, although both variants are usedin practice.The rule in Section 6.1.1 is based on t m = ∑ mk=1 l k, the rule in Section6.1.2 looks at individual eigenvalues l k , and the current rule, as appliedto PCA, uses l k−1 − l k as its criterion. There is, however, no formal numericalcut-off based on l k−1 − l k and, in fact, judgments of when l k−1 − l kstops being large (steep) will depend on the relative values of l k−1 − l kand l k − l k+1 ,aswellastheabsolute value of l k−1 − l k . Thus the rule isbased subjectively on the second, as well as the first, differences amongthe l k . Because of this, it is difficult to write down a formal numerical ruleand the procedure has until recently remained purely graphical. Tests thatattempt to formalize the procedure, due to Bentler and Yuan (1996,1998),are discussed in Section 6.1.4.Cattell’s formulation, where we look for the point at which l k−1 − l kbecomes fairly constant for several subsequent values, is perhaps less subjective,but still requires some degree of judgment. Both formulations ofthe rule seem to work well in practice, provided that there is a fairly sharp‘elbow,’ or change of slope, in the graph. However, if the slope graduallybecomes less steep, with no clear elbow, as in Figure 6.1, then it is clearlyless easy to use the procedure.A number of methods have been suggested in which the scree plot iscompared with a corresponding plot representing given percentiles, often a95 percentile, of the distributions of each variance (eigenvalue) when PCAis done on a ‘random’ matrix. Here ‘random’ usually refers to a correlationmatrix obtained from a random sample of n observations on p uncorrelatednormal random variables, where n, p are chosen to be the same as for thedata set of interest. A number of varieties of this approach, which goesunder the general heading parallel analysis, have been proposed in thepsychological literature. Parallel analysis dates back to Horn (1965), whereit was described as determining the number of factors in factor analysis.Its ideas have since been applied, sometimes inappropriately, to PCA.Most of its variants use simulation to construct the 95 percentiles empirically,and some examine ‘significance’ of loadings (eigenvectors), as wellas eigenvalues, using similar reasoning. Franklin et al. (1995) cite many ofthe most relevant references in attempting to popularize parallel analysisamongst ecologists. The idea in versions of parallel analysis that concentrateon eigenvalues is to retain m PCs, where m is the largest integer forwhich the scree graph lies above the graph of upper 95 percentiles. Boot-