Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

6.1. How Many Principal Components? 125needs to be estimated; an obvious estimate is the average of the (p − q)smallest eigenvalues of S.Besse and de Falguerolles (1993) start from the same fixed effects modeland concentrate on the special case just noted. They modify the lossfunction to becomeL q = 1 ∥∥P q −2ˆP∥ ∥∥2q , (6.1.9)where ˆP q = A q A ′ q, A q is the (p × q) matrix whose kth column is thekth eigenvalue of S, P q is the quantity corresponding to ˆP q for the trueq-dimensional subspace F q ,and‖.‖ denotes Euclidean norm. The loss functionL q measures the distance between the subspace F q and its estimateˆF q spanned by the columns of A q .The risk function that Besse and de Falguerolles (1993) seek to minimizeis R q = E[L q ]. As with f q , R q must be estimated, and Besse andde Falguerolles (1993) compare four computationally intensive ways of doingso, three of which were suggested by Besse (1992), building on ideasfrom Daudin et al. (1988, 1989). Two are bootstrap methods; one is basedon bootstrapping residuals from the q-dimensional model, while the otherbootstraps the data themselves. A third procedure uses a jackknife estimateand the fourth, which requires considerably less computational effort,constructs an approximation to the jackknife.Besse and de Falguerolles (1993) simulate data sets according to the fixedeffects model, with p = 10, q = 4 and varying levels of the noise varianceσ 2 . Because q and σ 2 are known, the true value of R q can be calculated.The four procedures outlined above are compared with the traditional screegraph and Kaiser’s rule, together with boxplots of scores for each principalcomponent. In the latter case a value m is sought such that the boxplotsare much less wide for components (m +1), (m +2),...,p than they arefor components 1, 2,...,m.As the value of σ 2 increases, all of the criteria, new or old, deteriorate intheir performance. Even the true value of R q does not take its minimumvalue at q = 4, although q = 4 gives a local minimum in all the simulations.Bootstrapping of residuals is uninformative regarding the value of q, butthe other three new procedures each have strong local minima at q =4.Allmethods have uninteresting minima at q =1andatq = p, but the jackknifetechniques also have minima at q =6, 7 which become more pronouncedas σ 2 increases. The traditional methods correctly choose q = 4 for smallσ 2 , but become less clear as σ 2 increases.The plots of the risk estimates are very irregular, and both Besse (1992)and Besse and de Falguerolles (1993) note that they reflect the importantfeature of stability of the subspaces retained. Many studies of stability (see,for example, Sections 10.2, 10.3, 11.1 and Besse, 1992) show that pairs ofconsecutive eigenvectors are unstable if their corresponding eigenvalues areof similar size. In a similar way, Besse and de Falguerolles’ (1993) risk