Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

6.1. How Many Principal Components? 127decide on m, two criteria need to be satisfied. First, the confidence intervalsfor λ m and λ m+1 should not overlap, and second no component shouldbe retained unless it has at least two coefficients whose confidence intervalsexclude zero. This second requirement is again relevant for factor analysis,but not PCA. With regard to the first criterion, it has already beennoted that avoiding small gaps between l m and l m+1 is desirable becauseit reduces the likelihood of instability in the retained components.6.1.6 Partial CorrelationFor PCA based on a correlation matrix, Velicer (1976) suggested that thepartial correlations between the p variables, given the values of the firstm PCs, may be used to determine how many PCs to retain. The criterionproposed is the average of the squared partial correlationsV =p∑i=1i≠jp∑j=1(r ∗ ij )2p(p − 1) ,where rij ∗ is the partial correlation between the ith and jth variables, giventhe first m PCs. The statistic rij ∗ is defined as the correlation between theresiduals from the linear regression of the ith variable on the first m PCs,and the residuals from the corresponding regression of the jth variable onthese m PCs. It therefore measures the strength of the linear relationshipbetween the ith and jth variables after removing the common effect of thefirst m PCs.The criterion V first decreases, and then increases, as m increases, andVelicer (1976) suggests that the optimal value of m corresponds to theminimum value of the criterion. As with Jackson’s (1993) bootstrap rulesof Section 6.1.5, and for the same reasons, this criterion is plausible asa means of deciding the number of factors in a factor analysis, but it isinappropriate in PCA. Numerous other rules have been suggested in thecontext of factor analysis (Reddon, 1984, Chapter 3). Many are subjective,although some, such as parallel analysis (see Sections 6.1.3, 6.1.5) attempta more objective approach. Few are relevant to, or useful for, PCA unlessthey are modified in some way.Beltrando (1990) gives a sketchy description of what appears to be anotherselection rule based on partial correlations. Instead of choosing m sothat the average squared partial correlation is minimized, Beltrando (1990)selects m for which the number of statistically significant elements in thematrix of partial correlations is minimized.6.1.7 Rules for an Atmospheric Science ContextAs mentioned in Section 4.3, PCA has been widely used in meteorologyand climatology to summarize data that vary both spatially and tempo-