Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

140 6. Choosing a Subset of Principal Components or Variablesone of McCabe’s four criteria when dealing with principal variables.Of the four criteria, McCabe (1984) argues that only for the first is itcomputationally feasible to explore all possible subsets, although the secondcan be used to define a stepwise variable-selection procedure; Bhargava andIshizuka (1991) describe such a procedure. The third and fourth criteria arenot explored further in McCabe’s paper.Several of the methods for selecting subsets of variables that preservemost of the information in the data associate variables with individual PCs.Cadima and Jolliffe (2001) extend the ideas of Cadima and Jolliffe (1995)for individual PCs, and look for subsets of variables that best approximatethe subspace spanned by a subset of q PCs, in the the sense that thesubspace spanned by the chosen variables is close to that spanned by thePCs of interest. A similar comparison of subspaces is the starting pointfor Besse and de Falguerolles’s (1993) procedures for choosing the numberof components to retain (see Section 6.1.5). In what follows we restrictattention to the first q PCs, but the reasoning extends easily to any set ofq PCs.Cadima and Jolliffe (2001) argue that there are two main ways of assessingthe quality of the subspace spanned by a subset of m variables. Thefirst compares the subspace directly with that spanned by the first q PCs;the second compares the data with its configuration when projected ontothe m-variable subspaces.Suppose that we wish to approximate the subspace spanned by the firstq PCs using a subset of m variables. The matrix of orthogonal projectionsonto that subspace is given by1P q =(n − 1) XS− q X ′ , (6.3.1)where S q = ∑ qk=1 l ka k a ′ kis the sum of the first q terms in the spectraldecomposition of S, andS − q = ∑ qk=1 l−1 k a ka ′ k is a generalized inverse of S q.The corresponding matrix of orthogonal projections onto the space spannedby a subset of m variables is1P m =(n − 1) XI mS −1m I ′ mX ′ , (6.3.2)where I m is the identity matrix of order m and S −1m is the inverse of the(m × m) submatrix of S corresponding to the m selected variables.The first measure of closeness for the two subspaces considered byCadima and Jolliffe (2001) is the matrix correlation between P q and P m ,defined bycorr(P q , P m )= √tr(P ′ qP m ). (6.3.3)tr(P ′ qP q )tr(P ′ mP m )This measure is also known as Yanai’s generalized coefficient of determination(Yanai, 1980). It was used by Tanaka (1983) as one of four criteria for