Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

13.5. Common Principal Components 359values of p, q and sample size in his tables closest to those of the presentexample. Hence, if Krzanowski’s tables are at all relevant for correlationmatrices, the sets of the first three PCs are not significantly different forthe three years 1982, 1983, 1984 as might be expected from such smallangles.If all three years are compared simultaneously, then the angles betweenthe subspaces formed by the first three PCs and the nearest vector to allthree subspaces are1982 1983 19841.17 ◦ 1.25 ◦ 0.67 ◦Again, the angles are very small; although no tables are available for assessingthe significance of these angles, they seem to confirm the impressiongiven by looking at the years two at a time that the sets of the first threePCs are not significantly different for the three years.Two points should be noted with respect to Krzanowski’s technique.First, it can only be used to compare subsets of PCs—if q = p, thenA 1p , A 2p will usually span p-dimensional space (unless either S 1 or S 2has zero eigenvalues), so that δ is necessarily zero. It seems likely that thetechnique will be most valuable for values of q that are small compared to p.The second point is that while δ is clearly a useful measure of the closenessof two subsets of PCs, the vectors and angles found from the second, third,..., eigenvalues and eigenvectors of A ′ 1qA 2q A ′ 2qA 1q are successively lessvaluable. The first two or three angles give an idea of the overall differencebetween the two subspaces, provided that q is not too small. However, if wereverse the analysis and look at the smallest eigenvalue and correspondingeigenvector of A ′ 1qA 2q A ′ 2qA 1q , then we find the maximum angle betweenvectors in the two subspaces (which will often be 90 ◦ , unless q is small).Thus, the last few angles and corresponding vectors need to be interpretedin a rather different way from that of the first few. The general problemof interpreting angles other than the first can be illustrated by again consideringthe first three PCs for the student anatomical data from 1982 and1983. We saw above that δ =2.02 ◦ , which is clearly very small; the secondand third angles for these data are 25.2 ◦ and 83.0 ◦ , respectively. Theseangles are fairly close to the 5% critical values given in Krzanowski (1982)for the second and third angles when p =8,q = 3 and the sample sizes areeach 50 (our data have p =7,q = 3 and sample sizes around 30), but it isdifficult to see what this result implies. In particular, the fact that the thirdangle is close to 90 ◦ might intuitively suggest that the first three PCs aresignificantly different for 1982 and 1983. Intuition is, however, contradictedby Krzanowski’s Table I, which shows that for sample sizes as small as 50(and, hence, certainly for samples of size 30), the 5% critical value for thethird angle is nearly 90 ◦ .Forq = 3 this is not particularly surprising—thedimension of A ′ 1qA 2q A ′ 2qA 1q is (3 × 3) so the third angle is the maximumangle between subspaces.