Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

11.2. Alternatives to Rotation 285Table 11.2. Hausmann’s 6-variable example: the first two PCs and constrainedcomponents.First component Second componentVariable PC Constrained PC ConstrainedSentence structure 0.43 1 −0.28 0Logical relationships 0.44 1 0.12 0Essay 0.32 1 −0.66 1Composition 0.46 1 −0.18 1Computation 0.39 1 0.53 −1Algebra 0.40 1 0.40 −1Percentage of total 74.16 74.11 16.87 16.66variationloadings chosen without worrying about the variances of the correspondingcomponents. Typically this is the set of vectors a k where a kk =1anda kj =0(k =1, 2,...,p; j =1, 2,...,p; j ≠ k), a kj being the jth elementof a k . A sequence of ‘simplicity-preserving’ transformations is then appliedto these vectors. Each transformation chooses a pair of the vectors and rotatesthem orthogonally in such a way that the variance associated with thecurrently higher variance component of the pair is increased at the expenseof the lower variance component. The algorithm stops when no non-trivialsimplicity-preserving transformation leads to an improvement in variance.Simplicity is achieved by considering a restricted set of angles for eachrotation. Only angles that result in the elements of the transformed vectorsbeing proportional to integers are allowed. Thus, ‘simple’ vectors aredefined in this technique as those whose elements are proportional to integers.It is usually the case that the transformed vector associated withthe higher variance tends to be simpler (proportional to smaller magnitudeintegers) than the other transformed vector. Cumulatively, this means thatwhen the algorithm terminates, all the vectors of loadings are simple, withthose for the first few components tending to be simpler than those forlater components. The choice of which pair of vectors to rotate at eachstage of the algorithm is that pair for which the increase in variance ofthe higher-variance component resulting from a simplicity-preserving rotationis maximized, although this strict requirement is relaxed to enablemore than one rotation of mutually exclusive pairs to be implementedsimultaneously.The algorithm for simple components includes a tuning parameter c,which determines the number of angles considered for each simplicitypreservingtransformation. This number is 2 c+2 for c =0, 1, 2,....Ascincreases, the simple components tend to become closer to the principalcomponents, but simplicity is sacrificed as the elements of the vectors ofloadings progressively become proportional to larger magnitude integers.