Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

9.3. Canonical Correlation Analysis and Related Techniques 229maximum covariance analysis. The reasoning behind the analysis, apartfrom being very easy to implement, is that two variables are more likelyto simultaneously have large loadings in the same high-variance combinedPC if they are strongly correlated. Thus, by looking at which variablesfrom different groups appear together in the same high-variance components,some ideas can be gained about the relationships between the twogroups. This is true to some extent, but the combined components do notdirectly quantify the precise form of the relationships, nor their strength,in the way that CCA or maximum covariance analysis does. One otherPCA-based technique considered by Bretherton et al. (1992) is to look atcorrelations between PCs of one set of variables and the variables themselvesfrom the other set. This takes us back to a collection of simple PCregressions.Another technique from Section 8.4, partial least squares (PLS), can begeneralized to the case of more than one response variable (Wold, 1984).Like single-response PLS, multiresponse PLS is often defined in terms of analgorithm, but Frank and Friedman (1993) give an interpretation showingthat multiresponse PLS successively maximizes the covariance between linearfunctions of the two sets of variables. It is therefore similar to maximumcovariance analysis, which was discussed in Section 9.3.3, but differs fromit in not treating response and predictor variables symmetrically. Whereasin maximum covariance analysis the vectors of coefficients of the linearfunctions are orthogonal within each set of variables, no such restrictionis placed on the response variables in multiresponse PLS. For the predictorvariables there is a restriction, but it is that the linear functions areuncorrelated, rather than having orthogonal vectors of coefficients.The standard technique when one set of variables consists of responsesand the other is made up of predictors is multivariate linear regression.Equation (8.1.1) generalizes toY = XB + E, (9.3.6)where Y, X are (n×p 1 ), (n×p 2 ) matrices of n observations on p 1 responsevariables and p 2 predictor variables, respectively, B is a (p 2 × p 1 ) matrix ofunknown parameters, and E is an (n × p 1 ) matrix of errors. The numberof parameters to be estimated is at least p 1 p 2 (as well as those in B, thereare usually some associated with the covariance matrix of the error term).Various attempts have been made to reduce this number by simplifying themodel. The reduced rank models of Davies and Tso (1982) form a generalclass of this type. In these models B is assumed to be of rank m