Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

14.2. Weights, Metrics, Transformations and Centerings 383The matrices A, L give, respectively, the eigenvectors and the square rootsof the eigenvalues of X ′ X, from which the coefficients and variances of thePCs for the sample covariance matrix S are easily found.In equation (14.2.1) we have U ′ U = I r , A ′ A = I r , where r is the rank ofX, andI r is the identity matrix of order r. Suppose now that Ω and Φ arespecified positive-definite symmetric matrices and that we replace (14.2.1)by a generalized SVDX = VMB ′ , (14.2.2)where V, B are (n×r), (p×r) matrices, respectively satisfying V ′ ΩV = I r ,B ′ ΦB = I r ,andM is a (r × r) diagonal matrix.This representation follows by finding the ordinary SVD of ˜X =Ω 1/2 XΦ 1/2 . If we write the usual SVD of ˜X as˜X = WKC ′ , (14.2.3)where K is diagonal, W ′ W = I r , C ′ C = I r , thenX = Ω −1/2 ˜XΦ−1/2= Ω −1/2 WKC ′ Φ −1/2 .Putting V = Ω −1/2 W, M = K, B = Φ −1/2 C gives (14.2.2), where M isdiagonal, V ′ ΩV = I r ,andB ′ ΦB = I r , as required. With this representation,Greenacre (1984) defines generalized PCs as having coefficients givenby the columns of B, in the case where Ω is diagonal. Rao (1964) suggesteda similar modification of PCA, to be used when oblique rather thanorthogonal axes are desired. His idea is to use the transformation Z = XB,where B ′ ΦB = I, for some specified positive-definite matrix, Φ; this ideaclearly has links with generalized PCA, as just defined.It was noted in Section 3.5 that, if we take the usual SVD and retainonly the first m PCs so that x ij is approximated bym˜x ij =m∑k=1u ik l 1/2ka jk (with notation as in Section 3.5),then m˜x ij provides a best possible rank m approximation to x ij in thesense of minimizing ∑ n ∑ pi=1 j=1 ( mx ij − x ij ) 2 among all possible rank mapproximations m x ij . It can also be shown (Greenacre, 1984, p. 39) that ifΩ, Φ are both diagonal matrices, with elements ω i ,i=1, 2,...,n; φ j ,j=1, 2,...,p, respectively, and if m˜x ij = ∑ mk=1 v ikm k b jk , where v ik , m k , b jkare elements of V, M, B defined in (14.2.2), then m˜x ij minimizesn∑ p∑ω i φ j ( m x ij − x ij ) 2 (14.2.4)i=1 j=1among all possible rank m approximations m x ij to x ij .Thus,thespecialcase of generalized PCA, in which Φ as well as Ω is diagonal, is