Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

228 9. Principal Components Used with Other Multivariate Techniquesbetween a ′ k2 x p2 and each of the variables x j . The difference between principalpredictors and redundancy analysis is that the principal predictorsare uncorrelated, whereas the derived variables in redundancy analysis arecorrelated but have vectors of loadings that are orthogonal. The presenceof correlation in redundancy analysis may be regarded as a drawback, andvan den Wollenberg (1977) suggests using the first few derived variablesfrom redundancy analysis as input to CCA. This will then produce uncorrelatedcanonical variates whose variances are unlikely to be small. Thepossibility of using the first few PCs from each set as input to CCA wasmentioned above, as was the disadvantage that excluded low-variance PCsmight contain strong inter-set correlations. As low-variance directions areunlikely to be of interest in redundancy analysis, using the first few PCsas input seems to be far safer in this case and is another option.It is of interest to note the similarity between equations (9.3.2), (9.3.3)and the eigenequation whose solution gives the loadings a k2 on x p2 forcanonical correlation analysis, namelyS xy S −1yy S yx a k2 = l k S xx a k2 , (9.3.4)using the present notation. Wang and Zwiers (2001) solve a version of(9.3.2) with covariance matrices replacing correlation matrices, by firstsolving the eigenequationS yx S −1xx S xy b k2 = l k b k2 , (9.3.5)and then setting a k2 = l − 1 2kS−1 xx S xy b k2 . This is equivalent to a PCA ofthe covariance matrix S yx S −1xx S xy of the predicted values of the responsevariables obtained from a multivariate regression on the predictor variables.Multivariate regression is discussed further in Section 9.3.4.Van den Wollenberg (1977) notes that PCA is a special case of redundancyanalysis (and principal predictors, but not CCA) when x p1 and x p2are the same (see also Property A6 in Chapter 2). Muller (1981) showsthat redundancy analysis is equivalent to orthogonally rotating the resultsof a multivariate regression analysis. DeSarbo and Jedidi (1986) give anumber of other properties, together with modifications and extensions, ofredundancy analysis.9.3.4 Other Techniques for Relating Two Sets of VariablesA number of other techniques for relating two sets of variables were notedin Section 8.4. They include separate PCAs on the two groups of variables,followed by the calculation of a regression equation to predict, again separately,each PC from one set from the PCs in the other set. Another way ofusing PCA is to concatenate the two sets of variables and find PCs for thecombined set of (p 1 + p 2 ) variables. This is sometimes known as combinedPCA, and is one of the methods that Bretherton et al. (1992) compare with