Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

396 14. Generalizations and Adaptations of Principal Component Analysisdescription here is brief. Stone and Porrill (2001) provide a more detailedintroduction.PCA has as its main objective the successive maximization of variance,and the orthogonality and uncorrelatedness constraints are extras, whichare included to ensure that the different components are measuring separatethings. By contrast, independent component analysis (ICA) takesthe ‘separation’ of components as its main aim. ICA starts from the viewthat uncorrelatedness is rather limited as it only considers a lack of linearrelationship, and that ideally components should be statistically independent.This is a stronger requirement than uncorrelatedness, with the twoonly equivalent for normal (Gaussian) random variables. ICA can thus beviewed as a generalization of PCA to non-normal data, which is the reasonfor including it in the present section. However this may lead to the mistakenbelief, as implied by Aires et al. (2000), that PCA assumes normality,which it does not. Aires and coworkers also describe PCA as assuming amodel in which the variables are linearly related to a set of underlyingcomponents, apart from an error term. This is much closer to the set-upfor factor analysis, and it is this ‘model’ that ICA generalizes.ICA assumes, instead of the factor analysis model x = Λf + e given inequation (7.1.1), that x = Λ(f), where Λ is some, not necessarily linear,function and the elements of f are independent. The components (factors)f are estimated by ˆf, which is a function of x. The family of functions fromwhich Λ can be chosen must be defined. As in much of the ICA literatureso far, Aires et al. (2000) and Stone and Porrill (2001) concentrateon the special case where Λ is restricted to linear functions. Within thechosen family, functions are found that minimize an ‘objective cost function,’based on information or entropy, which measures how far are theelements of ˆf from independence. This differs from factor analysis in thatthe latter has the objective of explaining correlations. Some details of a‘standard’ ICA method, including its entropy criterion and an algorithmfor implementation, are given by Stone and Porrill (2001).Typically, an iterative method is used to find the optimal ˆf, and likeprojection pursuit (see Section 9.2.2), a technique with which Stone andPorrill (2001) draw parallels, it is computationally expensive. As with projectionpursuit, PCA can be used to reduce dimensionality (use the first m,rather than all p) before starting the ICA algorithm, in order to reduce thecomputational burden (Aires et al., 2000; Stone and Porrill, 2001). It is alsosuggested by Aires and coworkers that the PCs form a good starting pointfor the iterative algorithm, as they are uncorrelated. These authors givean example involving sea surface temperature, in which they claim thatthe ICs are physically more meaningful than PCs. The idea that physicallymeaningful signals underlying a data set should be independent is a majormotivation for ICA. This is very different from the view taken in someapplications of factor analysis or rotated PCA, where it is believed that un-