Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

13.5. Common Principal Components 361draw distinctions between the properties of principal components found byeach. Although the different criteria lead to a number of different generalizations,it is arguable just how great a distinction should be drawn betweenthe three ungeneralized analyses (see Cadima and Jolliffe (1997); ten Bergeand Kiers (1997)).The first property considered by ten Berge and Kiers (1996) correspondsto Property A1 of Section 2.1, in which tr(B ′ ΣB) is minimized.For G groups of individuals treated separately this leads to minimizationof ∑ Gg=1 tr(B′ gΣ g B g ), but taking B g the same for each group givessimultaneous components that minimizeG∑G∑tr(B ′ Σ g B)=tr[B ′ ( Σ g )B]g=1g=1= G tr(B ′ ¯ΣB),where ¯Σ is the average of Σ 1 , Σ 2 ,...,Σ G .Ten Berge and Kiers’ (1996) second property is a sample version ofProperty A5 in Section 2.1. They express this property as minimizing‖X − XBC ′ ‖ 2 .ForG groups treated separately, the quantityG∑‖X g − X g B g C ′ g‖ 2 (13.5.3)g=1is minimized. Ten Berge and Kiers (1996) distinguish three different waysof adapting this formulation to find simultaneous components.• Minimize ∑ Gg=1 ‖X g − X g BC ′ g‖ 2 .• Minimize ∑ Gg=1 ‖X g − X g B g C ′ ‖ 2 .• Minimize (13.5.3) subject to Σ g B g = SD g , where D g is diagonal andS is a ‘common component structure.’The third optimality criterion considered by Ten Berge and Kiers (1996)is that noted at the end of Section 2.1, and expressed by Rao (1964) asminimizing ‖Σ − ΣB(B ′ ΣB) −1 B ′ Σ‖. Ten Berge and Kiers (1996) writethis as minimizing ‖Σ − FF ′ ‖ 2 , which extends to G groups by minimizing∑ Gg=1 ‖Σ g − F g F ′ g‖ 2 . This can then be modified to give simultaneous componentsby minimizing ∑ Gg=1 ‖Σ g − FF ′ ‖ 2 . They show that this criterionand the criterion based on Property A1 are both equivalent to the secondof their generalizations derived from Property A5.Ten Berge and Kiers (1996) compare properties of the three generalizationsof Property A5, but do not reach any firm conclusions as to whichis preferred. They are, however, somewhat dismissive of Flury’s (1988)approach on the grounds that it has at its heart the simultaneous diagonalizationof G covariance matrices and ‘it is by no means granted that