Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

152 7. Principal Component Analysis and Factor AnalysisOf these three assumptions, the first is a standard assumption for errorterms in most statistical models, and the second is convenient and loses nogenerality. The third may not be true, but if it is not, (7.1.2) can be simplyadapted to become x = µ + Λf + e, where E[x] =µ. This modificationintroduces only a slight amount of algebraic complication compared with(7.1.2), but (7.1.2) loses no real generality and is usually adopted.(ii) E[ee ′ ]=Ψ (diagonal)E[fe ′ ]=0 (a matrix of zeros)E[ff ′ ]=I m (an identity matrix)The first of these three assumptions is merely stating that the error termsare uncorrelated which is a basic assumption of the factor model, namelythat all of x which is attributable to common influences is contained inΛf, ande j , e k , j ≠ k are therefore uncorrelated. The second assumption,that the common factors are uncorrelated with the specific factors, is alsoa fundamental one. However, the third assumption can be relaxed so thatthe common factors may be correlated (oblique) rather than uncorrelated(orthogonal). Many techniques in factor analysis have been developed forfinding orthogonal factors, but some authors, such as Cattell (1978, p.128), argue that oblique factors are almost always necessary in order toget a correct factor structure. Such details will not be explored here asthe present objective is to compare factor analysis with PCA, rather thanto give a full description of factor analysis, and for convenience all threeassumptions will be made.(iii) For some purposes, such as hypothesis tests to decide on an appropriatevalue of m, it is necessary to make distributional assumptions. Usuallythe assumption of multivariate normality is made in such cases but,as with PCA, many of the results of factor analysis do not depend onspecific distributional assumptions.(iv) Some restrictions are generally necessary on Λ, because without anyrestrictions there will be a multiplicity of possible Λs that give equallygood solutions. This problem will be discussed further in the nextsection.7.2 Estimation of the Factor ModelAt first sight, the factor model (7.1.2) looks like a standard regression modelsuch as that given in Property A7 of Section 3.1 (see also Chapter 8). However,closer inspection reveals a substantial difference from the standardregression framework, namely that neither Λ nor f in (7.1.2) is known,whereas in regression Λ would be known and f would contain the only unknownparameters. This means that different estimation techniques must