Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

194 8. Principal Components in Regression AnalysisTable 8.4. Variable selection using various techniques on the pitprop data. (Eachrow corresponds to a selected subset with × denoting a selected variable.)Variables1 2 3 4 5 6 7 8 9 10 11 12 13Five variablesJeffers (1981) × × × × ×× × × × ×McCabe (1982) × × × × ×× × × × ×× × × × ×× × × × ×Six variablesJeffers (1981) × × × × × ×McCabe (1982) × × × × × ×Jolliffe (1973) × × × × × ×× × × × × ×{× × × × × ×McCabe (1982)× × × × × ×Jolliffe (1973)× × × × × ×Eight variablesMansfield et al. × × × × × × × ×(1977)Boneh and × × × × × × × ×Mendieta (1994)ble. McCabe (1982), using a technique related to PCA (see Section 6.3),and with a similar purpose to Jolliffe’s (1973) methods, chose subsets ofvarious sizes. McCabe’s subsets in Table 8.4 are the best few with respectto a single criterion, whereas Jolliffe gives the single best subset butfor several different methods. The best subsets due to Jolliffe (1973) andMcCabe (1982) have considerable overlap with each other, but there aresubstantial differences from the subsets of Jeffers (1981) and Mansfield etal. (1977). This reflects the different aims of the different selection methods.It shows again that substantial variation within the set of regressor variablesdoes not necessarily imply any relationship with y and, conversely,that variables having little correlation with the first few PCs can still beimportant in predicting y. Interestingly, the subset chosen by Boneh andMendieta (1994) overlaps less with Jeffers’ (1981) selection than that ofMansfield et al. (1977), but more than do those of Jolliffe (1973) or McCabe(1982).