Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

3.8. Patterned Covariance and Correlation Matrices 57relations are positive and not close to zero. Sometimes a variable in such agroup will initially have entirely negative correlations with the other membersof the group, but the sign of a variable is often arbitrary, and switchingthe sign will give a group of the required structure. If correlations betweenthe q members of the group and variables outside the group are close tozero, then there will be q PCs ‘associated with the group’ whose coefficientsfor variables outside the group are small. One of these PCs will havea large variance, approximately 1 + (q − 1)¯r, where ¯r is the average correlationwithin the group, and will have positive coefficients for all variablesin the group. The remaining (q − 1) PCs will have much smaller variances(of order 1 − ¯r), and will have some positive and some negative coefficients.Thus the ‘large variance PC’ for the group measures, roughly, the averagesize of variables in the group, whereas the ‘small variance PCs’ give ‘contrasts’between some or all of the variables in the group. There may beseveral such groups of variables in a data set, in which case each group willhave one ‘large variance PC’ and several ‘small variance PCs.’ Conversely,as happens not infrequently, especially in biological applications when allvariables are measurements on individuals of some species, we may findthat all p variables are positively correlated. In such cases, the first PCis often interpreted as a measure of size of the individuals, whereas subsequentPCs measure aspects of shape (see Sections 4.1, 13.2 for furtherdiscussion).The discussion above implies that the approximate structure and variancesof the first few PCs can be deduced from a correlation matrix,provided that well-defined groups of variables are detected, including possiblysingle-variable groups, whose within-group correlations are high, andwhose between-group correlations are low. The ideas can be taken further;upper and lower bounds on the variance of the first PC can be calculated,based on sums and averages of correlations (Friedman and Weisberg, 1981;Jackson, 1991, Section 4.2.3). However, it should be stressed that althoughdata sets for which there is some group structure among variables are notuncommon, there are many others for which no such pattern is apparent.In such cases the structure of the PCs cannot usually be found withoutactually performing the PCA.3.8.1 ExampleIn many of the examples discussed in later chapters, it will be seen that thestructure of some of the PCs can be partially deduced from the correlationmatrix, using the ideas just discussed. Here we describe an example in whichall the PCs have a fairly clear pattern. The data consist of measurements ofreflexes at 10 sites of the body, measured for 143 individuals. As with theexamples discussed in Sections 3.3 and 3.4, the data were kindly suppliedby Richard Hews of Pfizer Central Research.