Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)
Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s) Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)
14.2. Weights, Metrics, Transformations and Centerings 391clear which of ‘sites’ and ‘species’ should be treated as ‘variables’ and whichas ‘observations.’ Another possibility is to centre with respect to sites, butnot species, in other words, carrying out an analysis with sites rather thanspecies as the variables. Buckland and Anderson (1985) analyse their datain this way.Yet another technique which has been suggested for analysing sometypes of site-species data is correspondence analysis (see, for example, Section5.4.1 and Gauch, 1982). As pointed out in Section 13.4, correspondenceanalysis has some similarity to Mandel’s approach, and hence to doublycentred PCA. In doubly centred PCA we analyse the residuals from anadditive model for row and column (site and species) effects, whereas incorrespondence analysis the residuals from a multiplicative (independence)model are considered.Both uncentred and doubly centred PCA perform eigenanalyses on matriceswhose elements are not covariances or correlations, but which canstill be viewed as measures of similarity or association between pairs ofvariables. Another technique in the same vein is proposed by Elmore andRichman (2001). Their idea is to find ‘distances’ between variables whichcan then be converted into similarities and an eigenanalysis done on theresulting similarity matrix. Although Elmore and Richman (2001) notea number of possible distance measures, they concentrate on Euclideandistance, so that the distance d jk between variables j and k is[ ∑ n(x ij − x ik ) 2] 1 2.i=1If D is largest of the p 2 d jk , the corresponding similarity matrix is definedto have elementss jk =1− d jkD .The procedure is referred to as PCA based on ES (Euclidean similarity).There is an apparent connection with principal coordinate analysis (Section5.2) but for ES-based PCA it is distances between variables, ratherthan between observations, that are analysed.The technique is only appropriate if the variables are all measured in thesame units—it makes no sense to compute a distance between a vector oftemperatures and a vector of heights, for example. Elmore and Richman(2001) report that the method does better at finding known ‘modes’ in adata set than PCA based on either a covariance or a correlation matrix.However, as with uncentred and doubly centred PCA, it is much less clearthan it is for PCA what is optimized by the technique, and hence it is moredifficult to know how to interpret its results.
392 14. Generalizations and Adaptations of Principal Component Analysis14.3 Principal Components in the Presence ofSecondary or Instrumental VariablesRao (1964) describes two modifications of PCA that involve what he calls‘instrumental variables.’ These are variables which are of secondary importance,but which may be useful in various ways in examining thevariables that are of primary concern. The term ‘instrumental variable’ isin widespread use in econometrics, but in a rather more restricted context(see, for example, Darnell (1994, pp. 197–200)).Suppose that x is, as usual, a p-element vector of primary variables,and that w is a vector of s secondary, or instrumental, variables. Rao(1964) considers the following two problems, described respectively as ‘principalcomponents of instrumental variables’ and ‘principal components...uncorrelated with instrumental variables’:(i) Find linear functions γ ′ 1w, γ ′ 2w,..., of w that best predict x.(ii) Find linear functions α ′ 1x, α ′ 2x,... with maximum variances that,as well as being uncorrelated with each other, are also uncorrelatedwith w.For (i), Rao (1964) notes that w may contain some or all of the elementsof x, and gives two possible measures of predictive ability, corresponding tothe trace and Euclidean norm criteria discussed with respect to PropertyA5 in Section 2.1. He also mentions the possibility of introducing weightsinto the analysis. The two criteria lead to different solutions to (i), oneof which is more straightforward to derive than the other. There is a superficialresemblance between the current problem and that of canonicalcorrelation analysis, where relationships between two sets of variables arealso investigated (see Section 9.3), but the two situations are easily seen tobe different. However, as noted in Sections 6.3 and 9.3.4, the methodologyof Rao’s (1964) PCA of instrumental variables has reappeared under othernames. In particular, it is equivalent to redundancy analysis (van den Wollenberg,1977) and to one way of fitting a reduced rank regression model(Davies and Tso, 1982).The same technique is derived by Esposito (1998). He projects the matrixX onto the space spanned by W, where X, W are data matrices associatedwith x, w, and then finds principal components of the projected data. Thisleads to an eigenequationS XW S −1WW S WXa k = l k a k ,which is the same as equation (9.3.5). Solving that equation leads to redundancyanalysis. Kazi-Aoual et al. (1995) provide a permutation test,using the test statistic tr(S WX S −1XX S XW) to decide whether there is anyrelationship between the x and w variables.
- Page 372 and 373: 13.1. Principal Component Analysis
- Page 374 and 375: 13.2. Analysis of Size and Shape 34
- Page 376 and 377: 13.2. Analysis of Size and Shape 34
- Page 378 and 379: 13.3. Principal Component Analysis
- Page 380 and 381: 13.3. Principal Component Analysis
- Page 382 and 383: 13.4. Principal Component Analysis
- Page 384 and 385: 13.4. Principal Component Analysis
- Page 386 and 387: 13.5. Common Principal Components 3
- Page 388 and 389: 13.5. Common Principal Components 3
- Page 390 and 391: 13.5. Common Principal Components 3
- Page 392 and 393: 13.5. Common Principal Components 3
- Page 394 and 395: 13.6. Principal Component Analysis
- Page 396 and 397: 13.6. Principal Component Analysis
- Page 398 and 399: 13.7. PCA in Statistical Process Co
- Page 400 and 401: 13.8. Some Other Types of Data 369A
- Page 402 and 403: 13.8. Some Other Types of Data 371d
- Page 404 and 405: 14Generalizations and Adaptations o
- Page 406 and 407: 14.1. Non-Linear Extensions of Prin
- Page 408 and 409: 14.1. Additive Principal Components
- Page 410 and 411: 14.1. Additive Principal Components
- Page 412 and 413: 14.1. Additive Principal Components
- Page 414 and 415: 14.2. Weights, Metrics, Transformat
- Page 416 and 417: 14.2. Weights, Metrics, Transformat
- Page 418 and 419: 14.2. Weights, Metrics, Transformat
- Page 420 and 421: 14.2. Weights, Metrics, Transformat
- Page 424 and 425: 14.3. PCs in the Presence of Second
- Page 426 and 427: 14.4. PCA for Non-Normal Distributi
- Page 428 and 429: 14.5. Three-Mode, Multiway and Mult
- Page 430 and 431: 14.5. Three-Mode, Multiway and Mult
- Page 432 and 433: 14.6. Miscellanea 401• Linear App
- Page 434 and 435: 14.6. Miscellanea 40314.6.3 Regress
- Page 436 and 437: 14.7. Concluding Remarks 405space o
- Page 438 and 439: Appendix AComputation of Principal
- Page 440 and 441: A.1. Numerical Calculation of Princ
- Page 442 and 443: A.1. Numerical Calculation of Princ
- Page 444 and 445: A.1. Numerical Calculation of Princ
- Page 446 and 447: ReferencesAguilera, A.M., Gutiérre
- Page 448 and 449: References 417Apley, D.W. and Shi,
- Page 450 and 451: References 419Benasseni, J. (1986b)
- Page 452 and 453: References 421Boik, R.J. (1986). Te
- Page 454 and 455: References 423Castro, P.E., Lawton,
- Page 456 and 457: References 425Cook, R.D. (1986). As
- Page 458 and 459: References 427Dempster, A.P., Laird
- Page 460 and 461: References 429Feeney, G.J. and Hest
- Page 462 and 463: References 431in Descriptive Multiv
- Page 464 and 465: References 433Gunst, R.F. and Mason
- Page 466 and 467: References 435Hocking, R.R., Speed,
- Page 468 and 469: References 437Jeffers, J.N.R. (1978
- Page 470 and 471: References 439Kazi-Aoual, F., Sabat
14.2. Weights, Metrics, Transformations and Centerings 391clear which of ‘sites’ and ‘species’ should be treated as ‘variables’ and whichas ‘observations.’ Another possibility is to centre with respect to sites, butnot species, in other words, carrying out an analysis with sites rather thanspecies as the variables. Buckland and Anderson (1985) analyse their datain this way.Yet another technique which has been suggested for analysing sometypes of site-species data is correspondence analysis (see, for example, Section5.4.1 and Gauch, 1982). As pointed out in Section 13.4, correspondenceanalysis has some similarity to Mandel’s approach, and hence to doublycentred PCA. In doubly centred PCA we analyse the residuals from anadditive model for row and column (site and species) effects, whereas incorrespondence analysis the residuals from a multiplicative (independence)model are considered.Both uncentred and doubly centred PCA perform eigenanalyses on matriceswhose elements are not covariances or correlations, but which canstill be viewed as measures of similarity or association between pairs ofvariables. Another technique in the same vein is proposed by Elmore andRichman (2001). Their idea is to find ‘distances’ between variables whichcan then be converted into similarities and an eigenanalysis done on theresulting similarity matrix. Although Elmore and Richman (2001) notea number of possible distance measures, they concentrate on Euclideandistance, so that the distance d jk between variables j and k is[ ∑ n(x ij − x ik ) 2] 1 2.i=1If D is largest of the p 2 d jk , the corresponding similarity matrix is definedto have elementss jk =1− d jkD .The procedure is referred to as PCA based on ES (Euclidean similarity).There is an apparent connection with principal coordinate analysis (Section5.2) but for ES-based PCA it is distances between variables, ratherthan between observations, that are analysed.The technique is only appropriate if the variables are all measured in thesame units—it makes no sense to compute a distance between a vector oftemperatures and a vector of heights, for example. Elmore and Richman(2001) report that the method does better at finding known ‘modes’ in adata set than PCA based on either a covariance or a correlation matrix.However, as with uncentred and doubly centred PCA, it is much less clearthan it is for PCA what is optimized by the technique, and hence it is moredifficult to know how to interpret its results.