Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

400 14. Generalizations and Adaptations of Principal Component Analysis14.6 MiscellaneaThis penultimate section discusses briefly some topics involving PCA thatdo not fit very naturally into any of the other sections of the book.14.6.1 Principal Components and Neural NetworksThis subject is sufficiently large to have a book devoted to it (Diamantarasand Kung, 1996). The use of neural networks to provide non-linearextensions of PCA is discussed in Section 14.1.3 and computational aspectsare revisited in Appendix A1. A few other related topics are notedhere, drawing mainly on Diamantaras and Kung (1996), to which the interestedreader is referred for further details. Much of the work in thisarea is concerned with constructing efficient algorithms, based on neuralnetworks, for deriving PCs. There are variations depending on whether asingle PC or several PCs are required, whether the first or last PCs areof interest, and whether the chosen PCs are found simultaneously or sequentially.The advantage of neural network algorithms is greatest whendata arrive sequentially, so that the PCs need to be continually updated.In some algorithms the transformation to PCs is treated as deterministic;in others noise is introduced (Diamantaras and Kung, 1996, Chapter 5). Inthis latter case, the components are written asy = B ′ x + e,and the original variables are approximated byˆx = Cy = CB ′ x + Ce,where B, C are (p × q) matrices and e is a noise term. When e = 0, minimizingE[(ˆx − x) ′ (ˆx − x)] with respect to B and C leads to PCA (thisfollows from Property A5 of Section 2.1), but the problem is complicatedby the presence of the term Ce in the expression for ˆx. Diamantaras andKung (1996, Chapter 5) describe solutions to a number of formulations ofthe problem of finding optimal B and C. Some constraints on B and/or Care necessary to make the problem well-defined, and the different formulationscorrespond to different constraints. All solutions have the commonfeature that they involve combinations of the eigenvectors of the covariancematrix of x with the eigenvectors of the covariance matrix of e. As withother signal/noise problems noted in Sections 12.4.3 and 14.2.2, there isthe necessity either to know the covariance matrix of e or to be able toestimate it separately from that of x.Networks that implement extensions of PCA are described in Diamantarasand Kung (1996, Chapters 6 and 7). Most have links to techniquesdeveloped independently in other disciplines. As well as non-linearextensions, the following analysis methods are discussed: