Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

14.1. Non-Linear Extensions of Principal Component Analysis 375and Marriott (1994, Chapter 8), and Michailidis and de Leeuw (1998) givea review.Gifi’s (1990) form of non-linear PCA is based on a generalization of theresult that if, for an (n × p) data matrix X, we minimizetr{(X − YB ′ ) ′ (X − YB ′ )}, (14.1.1)with respect to the (n × q) matrix Y whose columns are linear functions ofcolumns of X, and with respect to the (q ×p) matrix B ′ where the columnsof B are orthogonal, then the optimal Y consists of the values (scores) ofthe first q PCs for the n observations, and the optimal matrix B consistsof the coefficients of the first q PCs. The criterion (14.1.1) corresponds tothat used in the sample version of Property A5 (see Section 2.1), and canbe rewritten as⎧⎫⎨ p∑⎬tr (x j − Yb j ) ′ (x j − Yb j )⎩⎭ , (14.1.2)j=1where x j , b j are the jth columns of X, B ′ , respectively.Gifi’s (1990) version of non-linear PCA is designed for categorical variablesso that there are no immediate values of x j to insert in (14.1.2). Anyvariables that are continuous are first converted to categories; then valuesneed to be derived for each category of every variable. We can express thisalgebraically as the process minimizing⎧⎫⎨ p∑⎬tr (G j c j − Yb j ) ′ (G j c j − Yb j )⎩⎭ , (14.1.3)j=1where G j is an (n × g j ) indicator matrix whose (h, i)th value is unity ifthe hth observation is in the ith category of the jth variable and is zerootherwise, and c j is a vector of length g j containing the values assignedto the g j categories of the jth variable. The minimization takes place withrespect to both c j and Yb j , so that the difference from (linear) PCA isthat there is optimization over the values of the variables in addition tooptimization of the scores on the q components. The solution is found byan alternating least squares (ALS) algorithm which alternately fixes thec j and minimizes with respect to the Yb j , then fixes the Yb j at the newvalues and minimizes with respect to the c j , fixes the c j at the new valuesand minimizes over Yb j , and so on until convergence. This is implementedby the Gifi-written PRINCALS computer program (Gifi, 1990, Section 4.6)which is incorporated in the SPSS software.A version of non-linear PCA also appears in another guise within theGifi system. For two categorical variables we have a contingency table thatcan be analysed by correspondence analysis (Section 13.1). For more thantwo categorical variables there is an extension of correspondence analysis,called multiple correspondence analysis (see Section 13.1 and Greenacre,