Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

14.3. PCs in the Presence of Secondary or Instrumental Variables 393Inevitably, the technique has been generalized. For example, Sabatieret al. (1989) do so using the generalization of PCA described in Section14.2.2, with triples (X, Q 1 , D), (W, Q 2 , D). They note that Rao’s(1964) unweighted version of PCA of instrumental variables results fromdoing a generalized PCA on W, with D = 1 n I n,andQ 2 chosen to minimize‖XX ′ − WQ 2 W ′ ‖, where ‖.‖ denotes Euclidean norm. Sabatier et al.(1989) extend this to minimize ‖XQ 1 X ′ D − WQ 2 W ′ D‖ with respect toQ 2 . They show that for various choices of Q 1 and D, a number of otherstatistical techniques arise as special cases. Another generalization is givenby Takane and Shibayama (1991). For an (n 1 ×p 1 ) data matrix X, considerthe prediction of X not only from an (n 1 × p 2 ) matrix of additional variablesmeasured on the same individuals, but also from an (n 2 × p 1 ) matrixof observations on a different set of n 2 individuals for the same variables asin X. PCA of instrumental variables occurs as a special case when only thefirst predictor matrix is present. Takane et al. (1995) note that redundancyanalysis, and Takane and Shibayama’s (1991) extension of it, amount toprojecting the data matrix X onto a subspace that depends on the externalinformation W and then conducting a PCA on the projected data. Thisprojection is equivalent to putting constraints on the PCA, with the sameconstraints imposed in all dimensions. Takane et al. (1995) propose a furthergeneralization in which different constraints are possible in differentdimensions. The principal response curves of van den Brink and ter Braak(1999) (see Section 12.4.2) represent another extension.One situation mentioned by Rao (1964) in which problem type (ii)(principal components uncorrelated with instrumental variables) might berelevant is when the data x 1 , x 2 ,...,x n form a multiple time series with pvariables and n time points, and it is required to identify linear functionsof x that have large variances, but which are uncorrelated with ‘trend’in the time series (see Section 4.5 for an example where the first PC isdominated by trend). Rao (1964) argues that such functions can be foundby defining instrumental variables which represent trend, and then solvingthe problem posed in (ii), but he gives no example to illustrate thisidea. A similar idea is employed in some of the techniques discussed in Section13.2 that attempt to find components that are uncorrelated with anisometric component in the analysis of size and shape data. In the contextof neural networks, Diamantaras and Kung (1996, Section 7.1) describe aform of ‘constrained PCA’ in which the requirement of uncorrelatedness inRao’s method is replaced by orthogonality of the vectors of coefficients inthe constrained PCs to the subspace spanned by a set of constraints (seeSection 14.6.1).Kloek and Mennes (1960) also discussed the use of PCs as ‘instrumentalvariables,’ but in an econometric context. In their analysis, a number ofdependent variables y are to be predicted from a set of predictor variablesx. Information is also available concerning another set of variables w (theinstrumental variables) not used directly in predicting y, but which can