Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

13.5. Common Principal Components 357cases of interest such as identity or diagonal covariance matrices. Krzanowski(1990) and Flury (1995) also discuss the use of the common principalcomponent model in discriminant analysis (see Section 9.1).Flury and Neuenschwander (1995) look at the situation in which the assumptionthat the G groups of variables are independent is violated. Thiscan occur, for example, if the same variables are measured for the sameindividuals at G different times. They argue that, in such circumstances,when G = 2 the common PC model can provide a useful alternative tocanonical correlation analysis (CCA) ( see Section 9.3) for examining relationshipsbetween two groups and, unlike CCA, it is easily extended to thecase of G>2 groups. Neuenschwander and Flury (2000) discuss in detailthe theory underlying the common PC model for dependent groups.Krzanowski (1984a) describes a simpler method of obtaining estimatesof A and Λ g based on the fact that if (13.5.1) is true then the columns of Acontain the eigenvectors not only of Σ 1 , Σ 2 ,...,Σ G individually but of anylinear combination of Σ 1 , Σ 2 ,...,Σ G . He therefore uses the eigenvectorsof S 1 + S 2 + ...+ S G , where S g is the sample covariance matrix for the gthpopulation, to estimate A, and then substitutes this estimate and S g for Aand Σ g , respectively, in (13.5.1) to obtain estimates of Λ g ,g=1, 2,...,G.To assess whether or not (13.5.1) is true, the estimated eigenvectorsof S 1 + S 2 + ... + S G can be compared with those estimated for someother weighted sum of S 1 , S 2 ,...,S G chosen to have different eigenvectorsfrom S 1 + S 2 + ...+ S G if (13.5.1) does not hold. The comparison betweeneigenvectors can be made either informally or using methodology developedby Krzanowski (1979b), which is now described.Suppose that sample covariance matrices S 1 , S 2 are available for twogroups of individuals, and that we wish to compare the two sets of PCsfound from S 1 and S 2 . Let A 1q , A 2q be (p × q) matrices whose columnscontain the coefficients of the first q PCs based on S 1 , S 2 , respectively.Krzanowski’s (1979b) idea is to find the minimum angle δ between thesubspaces defined by the q columns of A 1q and A 2q , together with associatedvectors in these two subspaces that subtend this minimum angle.This suggestion is based on an analogy with the congruence coefficient,which has been widely used in factor analysis to compare two sets of factorloadings (Korth and Tucker, 1975) and which, for two vectors, can be interpretedas the cosine of the angle between those vectors. In Krzanowski’s(1979b) set-up, it turns out that δ is given byδ =cos −1 ( ν 1/21where ν 1 is the first (largest) eigenvalue of A ′ 1qA 2q A ′ 2qA 1q , and the vectorsthat subtend the minimum angle are related, in a simple way, to thecorresponding eigenvector.The analysis can be extended by looking at the second, third,. . . , eigenvaluesand corresponding eigenvectors of A ′ 1qA 2q A ′ 2qA 1q ; from these can be),