Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

258 10. Outlier Detection, Influential Observations and Robust Estimationlap between the four most influential observations for the first and secondeigenvalues in Table 10.2. However, for eigenvalues in a correlation matrix,more than one value is likely to be affected by a very influential observation,because the sum of eigenvalues remains fixed. Also, large changes inan eigenvector for either correlation or covariance matrices result in at leastone other eigenvector being similarly changed, because of the orthogonalityconstraints. These results are again reflected in Tables 10.2 and 10.3, withobservations appearing as influential for both of the first two eigenvectors,and for both eigenvalues in the case of the correlation matrix.Comparing the results for covariance and correlation matrices in Tables10.2 and 10.3, we see that several observations are influential for bothmatrices. This agreement occurs because, in the present example, the originalvariables all have similar variances, so that the PCs for correlationand covariance matrices are similar. In examples where the PCs based oncorrelation and covariance matrices are very different, the sets of influentialobservations for the two analyses often show little overlap.Turning now to the observations that have been identified as influentialin Table 10.3, we can examine their positions with respect to the first twoPCs on Figures 5.2 and 5.3. Observation 34, which is the most influentialobservation on eigenvalues 1 and 2 and on eigenvector 2, is the painter indicatedin the top left of Figure 5.2, Fr. Penni. His position is not particularlyextreme with respect to the first PC, and he does not have an unduly largeinfluence on its direction. However, he does have a strong influence on boththe direction and variance (eigenvalue) of the second PC, and to balance theincrease which he causes in the second eigenvalue there is a compensatorydecrease in the first eigenvalue. Hence, he is influential on that eigenvaluetoo. Observation 43, Rembrandt, is at the bottom of Figure 5.2 and, likeFr. Penni, has a direct influence on PC2 with an indirect but substantialinfluence on the first eigenvalue. The other two observations, 28 and 31,Caravaggio and Palma Vecchio, which are listed in Table 10.3 as being influentialfor the first eigenvalue, have a more direct effect. They are the twoobservations with the most extreme values on the first PC and appear atthe extreme left of Figure 5.2.Finally, the observations in Table 10.3 that are most influential on thefirst eigenvector, two of which also have large values of influence for thesecond eigenvector, appear on Figure 5.2 in the second and fourth quadrantsin moderately extreme positions.Student Anatomical MeasurementsIn the discussion of the data on student anatomical measurements in Section10.1 it was suggested that observation 16 is so extreme on the secondPC that it could be largely responsible for the direction of that component.Looking at influence functions for these data enables us to investigate thisconjecture. Not surprisingly, observation 16 is the most influential observa-