Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

154 7. Principal Component Analysis and Factor Analysischosen to maximizeQ =m∑k=1[ p∑j=1b 4 jk − 1 p( p∑j=1b 2 jk) 2]. (7.2.2)The terms in the square brackets are proportional to the variances ofsquared loadings for each rotated factor. In the usual implementations offactor analysis the loadings are necessarily between −1 and 1, so the criteriontends to drive squared loadings towards the end of the range 0 to 1,and hence loadings towards −1, 0 or 1 and away from intermediate values,as required. The quantity Q in equation (7.2.2) is the raw varimax criterion.A normalized version is also used in which b jk is replaced byb√ jk∑mk=1 b2 jkin (7.2.2).As discussed in Section 11.1, rotation can be applied to principal componentcoefficients in order to simplify them, as is done with factor loadings.The simplification achieved by rotation can help in interpreting the factorsor rotated PCs. This is illustrated nicely using diagrams (see Figures 7.1and 7.2) in the simple case where only m = 2 factors or PCs are retained.Figure 7.1 plots the loadings of ten variables on two factors. In fact, theseloadings are the coefficients a 1 , a 2 for the first two PCs from the examplepresented in detail later in the chapter, normalized so that a ′ k a k = l k ,where l k is the kth eigenvalue of S, rather than a ′ k a k = 1. When an orthogonalrotation method (varimax) is performed, the loadings for the rotatedfactors (PCs) are given by the projections of each plotted point onto theaxes represented by dashed lines in Figure 7.1.Similarly, rotation using an oblique rotation method (direct quartimin)gives loadings after rotation by projecting onto the new axes shown inFigure 7.2. It is seen that in Figure 7.2 all points lie close to one or otherof the axes, and so have near-zero loadings on the factor represented bythe other axis, giving a very simple structure for the loadings. The loadingsimplied for the rotated factors in Figure 7.1, whilst having simpler structurethan the original coefficients, are not as simple as those for Figure 7.2, thusillustrating the advantage of oblique, compared to orthogonal, rotation.Returning to the first stage in the estimation of Λ and Ψ, there is sometimesa problem with identifiability, meaning that the size of the data setis too small compared to the number of parameters to allow those parametersto be estimated (Jackson, 1991, Section 17.2.6; Everitt and Dunn,2001, Section 12.3)). Assuming that identifiability is not a problem, thereare a number of ways of constructing initial estimates (see, for example,Lewis-Beck (1994, Section II.2); Rencher (1998, Section 10.3); Everitt andDunn (2001, Section 12.2)). Some, such as the centroid method (see Cattell,1978, Section 2.3), were developed before the advent of computers and