Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

208 9. Principal Components Used with Other Multivariate Techniquesspectral decomposition corresponding to the first q g PCs. Thus it replacesin (9.1.2) byS −1gq g∑k=1l −1gk a gka ′ gk +p∑k=q g+1¯l−1 g a gk a ′ gk.Another difference between DASCO and SIMCA is that DASCO retainsthe log-determinant term in (9.1.2).In Frank and Freidman’s (1989) simulation studies and data analyses,SIMCA is outperformed by both DASCO and regularized discriminantanalysis in many circumstances, especially when the covariance structuresare different in different groups. This is perhaps not surprising, given itsomission of the log-determinant term from (9.1.2). The absence of the firstq g PCs from SIMCA’s measure of discrepancy of an observation from agroup also means that it is unlikely to do well when the groups differin the directions of these PCs (Mertens et al., 1994). These latter authorstreat SIMCA’s measure of discrepancy between an observation and a groupas an indication of the outlyingness of the observation with respect to thegroup, and suggest modifications of SIMCA in which other outlier detectionmeasures are used (see Section 10.1).A similar idea to SIMCA is suggested by Asselin de Beauville (1995).As with SIMCA, separate PCAs are done for each group, but here anobservation is assigned on the basis of a measure that combines the smallestdistance of the observation from an axis defining a PC for a group and itsscoreonthatPC.It has been noted above that discriminant analysis can be treated as amultiple regression problem, with dummy variables, corresponding to thegroup membership, as dependent variables. Other regression techniques,as well as PC regression, can therefore be adapted for the discriminantproblem. In particular, partial least squares (PLS — see Section 8.4), whichis, in a sense, a compromise between least squares and PC regression, canbe used in the discriminant context (Vong et al. (1990)).SIMCA calculates PCs separately within each group, compared with themore usual practice of finding PCs from the overall covariance matrix, orfrom a pooled within-group covariance matrix. Yet another type of PCcan be derived from the between-group covariance matrix S b . However,if the dominant direction of S b coincides with that of the within-groupcovariance matrix S w , there may be better discriminatory power in a differentdirection, corresponding to a low-variance direction for S w .Thisleads Devijver and Kittler (1982, Section 9.7) to ‘prewhiten’ the data,using S w , before finding PCs with respect to S b . This is equivalent tothe well-known procedure of canonical variate analysis or canonical discriminantanalysis, in which uncorrelated linear functions are found thatdiscriminate as well as possible between the groups. Canonical variates aredefined as γ ′ 1x, γ ′ 2x,...,γ ′ g−1x where γ ′ k x maximizes the ratio γ′ S b γγ ′ S wγ ,of