Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

386 14. Generalizations and Adaptations of Principal Component Analysis14.2.2 MetricsThe idea of defining PCA with respect to a metric or an inner-product datesback at least to Dempster (1969, Section 7.6). Following the publication ofCailliez and Pagès (1976) it became, together with an associated ‘dualitydiagram,’ a popular view of PCA in France in the 1980s (see, for example,Caussinus, 1986; Escoufier, 1987). In this framework, PCA is defined interms of a triple (X, Q, D), the three elements of which are:• the matrix X is the (n × p) data matrix, which is usually but notnecessarily column-centred;• the (p × p) matrix Q defines a metric on the p variables, so that thedistance between two observations x j and x k is (x j −x k ) ′ Q(x j −x k );• the (n × n) matrix D is usually diagonal, and its diagonal elementsconsist of a set of weights for the n observations. It can, however, bemore general, for example when the observations are not independent,as in time series (Caussinus, 1986; Escoufier, 1987).The usual definition of covariance-based PCA has Q = I p the identitymatrix, and D = 1 n I n, though to get the sample covariance matrix withdivisor (n − 1) it is necessary to replace n by (n − 1) in the definition ofD, leading to a set of ‘weights’ which do not sum to unity. CorrelationbasedPCA is achieved either by standardizing X, or by taking Q to bethe diagonal matrix whose jth diagonal element is the reciprocal of thestandard deviation of the jth variable, j =1, 2,...,p.Implementation of PCA with a general triple (X, Q, D) is readilyachieved by means of the generalized SVD, described in Section 14.2.1,with Φ and Ω from that section equal to Q and D from this section. Thecoefficients of the generalized PCs are given in the columns of the matrixB defined by equation (14.2.2). Alternatively, they can be found from aneigenanalysis of X ′ DXQ or XQX ′ D (Escoufier, 1987).A number of particular generalizations of the standard form of PCA fitwithin this framework. For example, Escoufier (1987) shows that, in additionto the cases already noted, it can be used to: transform variables; toremove the effect of an observation by putting it at the origin; to look atsubspaces orthogonal to a subset of variables; to compare sample and theoreticalcovariance matrices; and to derive correspondence and discriminantanalyses. Maurin (1987) examines how the eigenvalues and eigenvectors ofa generalized PCA change when the matrix Q in the triple is changed.The framework also has connections with the fixed effects model of Section3.9. In that model, the observations x i are such that x i = z i + e i ,where z i lies in a q-dimensional subspace and e i is an error term with zeromean and covariance matrix σ2w iΓ. Maximum likelihood estimation of themodel, assuming a multivariate normal distribution for e, leads to a generalizedPCA, where D is diagonal with elements w i and Q (which is denoted