Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

202 9. Principal Components Used with Other Multivariate TechniquesFigure 9.1. Two data sets whose direction of separation is the same as that ofthe first (within-group) PC.do so) then omitting the low-variance PCs may actually throw away mostof the information in x concerning between-group variation.The problem is essentially the same one that arises in PC regressionwhere, as discussed in Section 8.2, it is inadvisable to look only at highvariancePCs, as the low-variance PCs can also be highly correlated with thedependent variable. That the same problem arises in both multiple regressionand discriminant analysis is hardly surprising, as linear discriminantanalysis can be viewed as a special case of multiple regression in whichthe dependent variable is a dummy variable defining group membership(Rencher, 1995, Section 8.3).An alternative to finding PCs from the within-group covariance matrixis mentioned by Rao (1964) and used by Chang (1983), Jolliffe etal. (1996) and Mager (1980b), among others. It ignores the group structureand calculates an overall covariance matrix based on the raw data. If thebetween-group variation is much larger than within-group variation, thenthe first few PCs for the overall covariance matrix will define directions in