Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

8.4. Variations on Principal Component Regression 183Another variation on the idea of PC regression has been used in severalmeteorological examples in which a multivariate (rather than multiple)regression analysis is appropriate, that is, where there are several dependentvariables as well as regressor variables. Here PCA is performed on thedependent variables and, separately, on the predictor variables. A numberof PC regressions are then carried out with, as usual, PCs of predictorvariables in place of the predictor variables but, in each regression, thedependent variable is now one of the high variance PCs of the original setof dependent variables. Preisendorfer and Mobley (1988, Chapter 9) discussthis set-up in some detail, and demonstrate links between the results andthose of canonical correlation analysis (see Section 9.3) on the two setsof variables. Briffa et al. (1986) give an example in which the dependentvariables are mean sea level pressures at 16 grid-points centred on theUK and France, and extending from 45 ◦ –60 ◦ N, and from 20 ◦ W–10 ◦ E. Thepredictors are tree ring widths for 14 oak ring width chronologies from theUK and Northern France. They transform the relationships found betweenthe two sets of PCs back into relationships between the original sets ofvariables and present the results in map form.The method is appropriate if the prediction of high-variance PCs of thedependent variables is really of interest, in which case another possibilityis to regress PCs of the dependent variables on the original predictorvariables. However, if overall optimal prediction of linear functions of dependentvariables from linear functions of predictor variables is required,then canonical correlation analysis (see Section 9.3; Mardia et al., 1979,Chapter 10; Rencher, 1995, Chapter 11) is more suitable. Alternatively,if interpretable relationships between the original sets of dependent andpredictor variables are wanted, then multivariate regression analysis or arelated technique (see Section 9.3; Mardia et al., 1979, Chapter 6; Rencher,1995, Chapter10) may be the most appropriate technique.The so-called PLS (partial least squares) method provides yet anotherbiased regression approach with links to PC regression. The method hasa long and complex history and various formulations (Geladi, 1988; Wold,1984). It has often been expressed only in terms of algorithms for its implementation,which makes it difficult to understand exactly what it does.A number of authors, for example, Garthwaite (1994) and Helland (1988,1990), have given interpretations that move away from algorithms towardsa more model-based approach, but perhaps the most appealing interpretationfrom a statistical point of view is that given by Stone and Brooks(1990). They show that PLS is equivalent to successively finding linearfunctions of the predictor variables that have maximum covariance withthe dependent variable, subject to each linear function being uncorrelatedwith previous ones. Whereas PC regression in concerned with variances derivedfrom X, and least squares regression maximizes correlations betweeny and X, PLS combines correlation and variance to consider covariance.Stone and Brooks (1990) introduce a general class of regression procedures,