Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

10.2. Influential Observations in a Principal Component Analysis 249influential depends on the analysis being done on the data set; observationsthat are influential for one type of analysis or parameter of interest maynot be so for a different analysis or parameter. This behaviour is evident inPCA where observations that are influential for the coefficients of a PC arenot necessarily influential for the variance of that PC, and vice versa. Wehave seen in the Section 10.1 that PCA can be used to search for influentialobservations in a regression analysis. The present section concentrateson looking for observations that are influential for some aspect of a PCA,either the variances, the coefficients (loadings) or the PCs themselves (thePC scores).The intuitive definition of the influence of an observation on a statistic,such as the kth eigenvalue l k or eigenvector a k of a sample covariance matrix,is simply the change in l k or a k , perhaps renormalized in some way,when the observation is deleted from the sample. For example, the sampleinfluence function for the ith observation on a quantity ˆθ, which might bel k or a k , is defined by Critchley (1985) as (n − 1)(ˆθ − ˆθ (i) ), where n is thesample size and ˆθ (i) is the quantity corresponding to ˆθ when the ith observationis omitted from the sample. Gnanadesikan and Kettenring (1972)suggested similar leave-one-out statistics for the correlation coefficient andfor ∑ pk=1 l k. The problems with influence defined in this manner are: theremay be no closed form for the influence function; the influence needs tobe computed afresh for each different sample. Various other definitions ofsample influence have been proposed (see, for example, Cook and Weisberg,1982, Section 3.4; Critchley, 1985); some of these have closed-form expressionsfor regression coefficients (Cook and Weisberg, 1982, Section 3.4), butnot for the statistics of interest in PCA. Alternatively, a theoretical influencefunction may be defined that can be expressed as a once-and-for-allformula and, provided that sample sizes are not too small, can be used toestimate the influence of actual and potential observations within a sample.To define the theoretical influence function, suppose that y is a p-variaterandom vector, and let y have cumulative distribution function (c.d.f.)F (y). If θ is a vector of parameters of the distribution of y (such as λ k , α k ,respectively, the kth eigenvalue and eigenvector of the covariance matrix ofy) then θ can be written as a functional of F .NowletF (y) be perturbedto become˜F (y) =(1− ε)F (y)+εδ x ,where 0