Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

7.1. Models for Factor Analysis 151may be estimated and how PCs are, but should perhaps not be, used inthis estimation process. Section 7.3 contains further discussion of differencesand similarities between PCA and factor analysis, and Section 7.4 gives anumerical example, which compares the results of PCA and factor analysis.Finally, in Section 7.5, a few concluding remarks are made regarding the‘relative merits’ of PCA and factor analysis, and the possible use of rotationwith PCA. The latter is discussed further in Chapter 11.7.1 Models for Factor AnalysisThe basic idea underlying factor analysis is that p observed random variables,x, can be expressed, except for an error term, as linear functionsof m (< p) hypothetical (random) variables or common factors, thatis if x 1 ,x 2 ,...,x p are the variables and f 1 ,f 2 ,...,f m are the factors,thenx 1 = λ 11 f 1 + λ 12 f 2 + ...+ λ 1m f m + e 1 (7.1.1)x 2 = λ 21 f 1 + λ 22 f 2 + ...+ λ 2m f m + e 2.x p = λ p1 f 1 + λ p2 f 2 + ...+ λ pm f m + e pwhere λ jk ,j=1, 2,...,p; k =1, 2,...,m are constants called the factorloadings, ande j ,j=1, 2,...,p are error terms, sometimes called specificfactors (because e j is ‘specific’ to x j , whereas the f k are ‘common’ to severalx j ). Equation (7.1.1) can be rewritten in matrix form, with obviousnotation, asx = Λf + e. (7.1.2)One contrast between PCA and factor analysis is immediately apparent.Factor analysis attempts to achieve a reduction from p tom dimensions by invoking a model relating x 1 ,x 2 ,...,x p to m hypotheticalor latent variables. We have seen in Sections 3.9, 5.3 and6.1.5 that models have been postulated for PCA, but for most practicalpurposes PCA differs from factor analysis in having no explicitmodel.The form of the basic model for factor analysis given in (7.1.2) is fairlystandard, although some authors give somewhat different versions. For example,there could be three terms on the right-hand side correspondingto contributions from common factors, specific factors and measurementerrors (Reyment and Jöreskog, 1993, p. 36), or the model could be madenon-linear. There are a number of assumptions associated with the factormodel, as follows:(i) E[e] =0, E[f] =0, E[x] =0.