Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

10.2. Influential Observations in a Principal Component Analysis 257changes occur. The ‘estimated’ changes in eigenvalues given in Tables 10.2and 10.3 are derived from multiples of I(·) in (10.2.2), (10.2.3), respectively,with the value of the kth PC for each individual observation substituted forz k , and with l k , a kj , r ij replacing λ k , α kj , ρ ij . The multiples are requiredbecause Change = Influence × ε, and we need a replacement for ε; here wehave used 1/(n − 1), where n = 54 is the sample size. Thus, apart from amultiplying factor (n − 1) −1 , ‘actual’ and ‘estimated’ changes are sampleand empirical influence functions, respectively.In considering changes to an eigenvector, there are changes to each ofthe p (= 4) coefficients in the vector. Comparing vectors is more difficultthan comparing scalars, but Tables 10.2 and 10.3 give the sum of squaresof changes in the individual coefficients of each vector, which is a plausiblemeasure of the difference between two vectors. This quantity is a monotonicallyincreasing function of the angle in p-dimensional space betweenthe original and perturbed versions of a k , which further increases its plausibility.The idea of using angles between eigenvectors to compare PCs isdiscussed in a different context in Section 13.5.The ‘actual’ changes for eigenvectors again come from leaving out oneobservation at a time, recomputing and then comparing the eigenvectors,while the estimated changes are computed from multiples of sample versionsof the expressions (10.2.4) and (10.2.5) for I(x; α k ). The changes ineigenvectors derived in this way are much smaller in absolute terms thanthe changes in eigenvalues, so the eigenvector changes have been multipliedby 10 3 in Tables 10.2 and 10.3 in order that all the numbers are of comparablesize. As with eigenvalues, apart from a common multiplier we arecomparing empirical and sample influences.The first comment to make regarding the results given in Tables 10.2 and10.3 is that the estimated values are extremely good in terms of obtainingthe correct ordering of the observations with respect to their influence.There are some moderately large discrepancies in absolute terms for the observationswith the largest influences, but experience with this and severalother data sets suggests that the most influential observations are correctlyidentified unless sample sizes become very small. The discrepancies in absolutevalues can also be reduced by taking multiples other than (n − 1) −1and by including second order (ε 2 ) terms.A second point is that the observations which are most influential fora particular eigenvalue need not be so for the corresponding eigenvector,and vice versa. For example, there is no overlap between the four mostinfluential observations for the first eigenvalue and its eigenvector in eitherthe correlation or covariance matrix. Conversely, observations can sometimeshave a large influence on both an eigenvalue and its eigenvector (seeTable 10.3, component 2, observation 34).Next, note that observations may be influential for one PC only, or affecttwo or more. An observation is least likely to affect more than one PC inthe case of eigenvalues for a covariance matrix—indeed there is no over-