Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

14.4. PCA for Non-Normal Distributions 395variables, but linear functions of them. Jensen (1997) shows that when‘scatter’ is defined as variance and x has a multivariate normal distribution,then his principal variables turn out to be the principal components. Thisresult is discussed following the spectral decomposition of the covariancematrix (Property A3) in Section 2.1. Jensen (1997) greatly extends theresult by showing that for a family of elliptical distributions and for awide class of definitions of scatter, his principal variables are the same asprincipal components.An idea which may be considered an extension of PCA to non-normaldata is described by Qian et al. (1994). They investigate linear transformationsof the p-variable vector x to q (< p) derived variables y thatminimize what they call an index of predictive power. This index is basedon minimum description length or stochastic complexity (see, for example,Rissanen and Yu (2000)) and measures the difference in stochastic complexitybetween x and y. The criterion is such that the optimal choice ofy depends on the probability distribution of x, and Qian and coworkers(1994) show that for multivariate normal x, the derived variables y arethe first q PCs. This can be viewed as an additional property of PCA, butconfusingly they take it as a definition of principal components. This leadsto their ‘principal components’ being different from the usual principalcomponents when the distribution of x is nonnormal. They discuss variousproperties of their components and include a series of tests of hypothesesfor deciding how many components are needed to adequately represent allthe original variables.Another possible extension of PCA to non-normal data is hinted at byO’Hagan (1994, Section 2.15). For a multivariate normal distribution, thecovariance matrix is given by the negative of the inverse ‘curvature’ of thelog-probability density function, where ‘curvature’ is defined as the matrixof second derivatives with respect to the elements of x. In the Bayesian setupwhere x is replaced by a vector of parameters θ, O’Hagan (1994) refersto the curvature evaluated at the modal value of θ as the modal dispersionmatrix. He suggests finding eigenvectors, and hence principal axes, basedon this matrix, which is typically not the covariance matrix for non-normaldistributions.14.4.1 Independent Component AnalysisThe technique, or family of techniques, known as independent componentanalysis (ICA) has been the subject of a large amount of research, startingin the late 1980s, especially in signal processing. It has been applied tovarious biomedical and imaging problems, and is beginning to be used inother fields such as atmospheric science. By the end of the 1990s it hadits own annual workshops and at least one book (Lee, 1998). Althoughit is sometimes presented as a competitor to PCA, the links are not particularlystrong-as we see below it seems closer to factor analysis—so the