Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

6.1. How Many Principal Components? 123the fact that a fixed sample covariance matrix S can result from differentdata matrices X. In addition to this two-tiered variability, there are manyparameters that can vary: n, p, and particularly the structure of Σ. Thismeans that simulation studies can only examine a fraction of the possibleparameter values, and are therefore of restricted applicability. Krzanowski(1983) looks at several different types of structure for Σ, and reaches theconclusion that W chooses about the right number of PCs in each case, althoughthere is a tendency for m to be too small. Wold (1978) also found,in a small simulation study, that R retains too few PCs. This underestimationfor m can clearly be overcome by moving the cut-offs for W and R,respectively, slightly below and slightly above unity. Although the cut-offsat R =1andW = 1 seem sensible, the reasoning behind them is not rigid,and they could be modified slightly to account for sampling variation in thesame way that Kaiser’s rule (Section 6.1.2) seems to work better when l ∗is changed to a value somewhat below unity. In later papers (Krzanowski,1987a; Krzanowski and Kline, 1995) a threshold for W of 0.9 is used.Krzanowski and Kline (1995) investigate the use of W in the context offactor analysis, and compare the properties and behaviour of W with threeother criteria derived from PRESS(m). Criterion P is based on the ratioP ∗ onand R (different from Wold’s R) on(PRESS(1) − PRESS(m)),PRESS(m)(PRESS(0) − PRESS(m)),PRESS(m)(PRESS(m − 1) − PRESS(m))(PRESS(m − 1) − PRESS(m + 1)) .In each case the numerator and denominator of the ratio are divided byappropriate degrees of freedom, and in each case the value of m for whichthe criterion is largest gives the number of factors to be retained. On thebasis of two previously analysed psychological examples, Krzanowski andKline (1995) conclude that W and P ∗ select appropriate numbers of factors,whereas P and R are erratic and unreliable. As discussed later in thissection, selection in factor analysis needs rather different considerationsfrom PCA. Hence a method that chooses the ‘right number’ of factors mayselect too few PCs.Cross-validation of PCs is computationally expensive for large data sets.Mertens et al. (1995) describe efficient algorithms for cross-validation, withapplications to principal component regression (see Chapter 8) and in theinvestigation of influential observations (Section 10.2). Besse and Ferré(1993) raise doubts about whether the computational costs of criteria basedon PRESS(m) are worthwhile. Using Taylor expansions, they show that