Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

10.4. Robust Estimation of Principal Components 265tra (1995) proposes a new elementwise estimate, and compares it to Devlinet al.’s (1981) estimates via a simulation study whose structure is the sameas that of Devlin and co-workers. Maronna and Yohai (1998) review manyof the robust covariance estimators. As Croux and Haesbroek (2000) pointout, every new robust covariance matrix estimator has a new robust PCAmethod associated with it.Campbell (1980) also proposed a modification of his robust estimationtechnique in which the weights assigned to an observation, in estimatingthe covariance matrix, are no longer functions of the overall Mahalanobisdistance of each observation from the mean. Instead, when estimating thekth PC, the weights are a decreasing function of the absolute value ofthe score of each observation for the kth PC. As with most of the techniquestested by Devlin et al. (1981), the procedure is iterative and thealgorithm to implement it is quite complicated, with nine separate steps.However, Campbell (1980) and Matthews (1984) each give an example forwhich the technique works well, and as well as estimating the PCs robustly,both authors use the weights found by the technique to identify potentialoutliers.The discussion in Section 10.2 noted that observations need not be particularly‘outlying’ in order to be influential. Thus, robust estimation methodsthat give weights to each observation based on Mahalanobis distance will‘miss’ any influential observations that are not extreme with respect to Mahalanobisdistance. It would seem preferable to downweight observationsaccording to their influence, rather than their Mahalanobis distance. As yet,no systematic work seems to have been done on this idea, but it should benoted that the influence function for the kth eigenvalue of a covariancematrix is an increasing function of the absolute score on the kth PC (seeequation (10.2.2)). The weights used in Campbell’s (1980) procedure thereforedownweight observations according to their influence on eigenvalues(though not eigenvectors) of the covariance (but not correlation) matrix.It was noted above that Devlin et al. (1981) and Campbell (1980) useM-estimators in some of their robust PCA estimates. A number of otherauthors have also considered M-estimation in the context of PCA. For example,Daigle and Rivest (1992) use it to construct a robust version of thebiplot (see Section 5.3). Ibazizen (1986) gives a thorough discussion of anM-estimation approach to robust PCA in which the definition of the firstrobust PC is based on a robust version of Property A5 in Section 2.1. Thisproperty implies that the first PC minimizes the sum of residual variancesarising from predicting each variable from the same single linear function ofthe variables. Each variance is an expected squared difference between thevariable and its mean. To robustify this quantity the mean is replaced bya robust estimator of location, and the square of the difference is replacedby another function, namely one of those typically chosen in M-estimation.Ibazizen (1986) includes a substantial amount of underlying theory for thisrobust PC, together with details of an algorithm to implement the proce-