Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

162 7. Principal Component Analysis and Factor Analysisduring their second term in school. Their average age at the time of testingwas 5 years, 5 months. Similar data sets are analysed in Lawley and Maxwell(1971).Table 7.1 gives the variances and the coefficients of the first four PCs,when the analysis is done on the correlation matrix. It is seen that thefirst four components explain nearly 76% of the total variation, and thatthe variance of the fourth PC is 0.71. The fifth PC, with a variance of0.51, would be discarded by most of the rules described in Section 6.1and, indeed, in factor analysis it would be more usual to keep only two,or perhaps three, factors in the present example. Figures 7.1, 7.2 earlier inthe chapter showed the effect of rotation in this example when only twoPCs are considered; here, where four PCs are retained, it is not possible toeasily represent the effect of rotation in the same diagrammatic way.All of the correlations between the ten variables are positive, so thefirst PC has the familiar pattern of being an almost equally weighted‘average’ of all ten variables. The second PC contrasts the first five variableswith the final five. This is not unexpected as these two sets ofvariables are of different types, namely ‘verbal’ tests and ‘performance’tests, respectively. The third PC is mainly a contrast between variables6 and 9, which interestingly were at the time the only two ‘new’ tests inthe WPSSI battery, and the fourth does not have a very straightforwardinterpretation.Table 7.2 gives the factor loadings when the first four PCs are rotatedusing an orthogonal rotation method (varimax), and an oblique method(direct quartimin). It would be counterproductive to give more varietiesof factor analysis for this single example, as the differences in detail tendto obscure the general conclusions that are drawn below. Often, resultsare far less sensitive to the choice of rotation criterion than to the choiceof how many factors to rotate. Many further examples can be found intexts on factor analysis such as Cattell (1978), Lawley and Maxwell (1971),Lewis-Beck (1994) and Rummel (1970).In order to make comparisons between Table 7.1 and Table 7.2 straightforward,the sum of squares of the PC coefficients and factor loadings arenormalized to be equal to unity for each factor. Typically, the output fromcomputer packages that implement factor analysis uses the normalization inwhich the sum of squares of coefficients in each PC before rotation is equalto the variance (eigenvalue) associated with that PC (see Section 2.3). Thelatter normalization is used in Figures 7.1 and 7.2. The choice of normalizationconstraints is important in rotation as it determines the propertiesof the rotated factors. Detailed discussion of these properties in the contextof rotated PCs is given in Section 11.1.The correlations between the oblique factors in Table 7.2 are given inTable 7.3 and it can be seen that there is a non-trivial degree of correlationbetween the factors given by the oblique method. Despite this, the structureof the factor loadings is very similar for the two factor rotation methods.