Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)
Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s) Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)
3.8. Patterned Covariance and Correlation Matrices 57relations are positive and not close to zero. Sometimes a variable in such agroup will initially have entirely negative correlations with the other membersof the group, but the sign of a variable is often arbitrary, and switchingthe sign will give a group of the required structure. If correlations betweenthe q members of the group and variables outside the group are close tozero, then there will be q PCs ‘associated with the group’ whose coefficientsfor variables outside the group are small. One of these PCs will havea large variance, approximately 1 + (q − 1)¯r, where ¯r is the average correlationwithin the group, and will have positive coefficients for all variablesin the group. The remaining (q − 1) PCs will have much smaller variances(of order 1 − ¯r), and will have some positive and some negative coefficients.Thus the ‘large variance PC’ for the group measures, roughly, the averagesize of variables in the group, whereas the ‘small variance PCs’ give ‘contrasts’between some or all of the variables in the group. There may beseveral such groups of variables in a data set, in which case each group willhave one ‘large variance PC’ and several ‘small variance PCs.’ Conversely,as happens not infrequently, especially in biological applications when allvariables are measurements on individuals of some species, we may findthat all p variables are positively correlated. In such cases, the first PCis often interpreted as a measure of size of the individuals, whereas subsequentPCs measure aspects of shape (see Sections 4.1, 13.2 for furtherdiscussion).The discussion above implies that the approximate structure and variancesof the first few PCs can be deduced from a correlation matrix,provided that well-defined groups of variables are detected, including possiblysingle-variable groups, whose within-group correlations are high, andwhose between-group correlations are low. The ideas can be taken further;upper and lower bounds on the variance of the first PC can be calculated,based on sums and averages of correlations (Friedman and Weisberg, 1981;Jackson, 1991, Section 4.2.3). However, it should be stressed that althoughdata sets for which there is some group structure among variables are notuncommon, there are many others for which no such pattern is apparent.In such cases the structure of the PCs cannot usually be found withoutactually performing the PCA.3.8.1 ExampleIn many of the examples discussed in later chapters, it will be seen that thestructure of some of the PCs can be partially deduced from the correlationmatrix, using the ideas just discussed. Here we describe an example in whichall the PCs have a fairly clear pattern. The data consist of measurements ofreflexes at 10 sites of the body, measured for 143 individuals. As with theexamples discussed in Sections 3.3 and 3.4, the data were kindly suppliedby Richard Hews of Pfizer Central Research.
58 3. Properties of Sample Principal ComponentsTable 3.4. Correlation matrix for ten variables measuring reflexes.V1 V2 V3 V4 V5 V6 V7 V8 V9 V10V1 1.00V2 0.98 1.00V3 0.60 0.62 1.00V4 0.71 0.73 0.88 1.00V5 0.55 0.57 0.61 0.68 1.00V6 0.55 0.57 0.56 0.68 0.97 1.00V7 0.38 0.40 0.48 0.53 0.33 0.33 1.00V8 0.25 0.28 0.42 0.47 0.27 0.27 0.90 1.00V9 0.22 0.21 0.19 0.23 0.16 0.19 0.40 0.41 1.00V10 0.20 0.19 0.18 0.21 0.13 0.16 0.39 0.40 0.94 1.00The correlation matrix for these data is given in Table 3.4, and thecoefficients of, and the variation accounted for by, the corresponding PCsare presented in Table 3.5. It should first be noted that the ten variablesfall into five pairs. Thus, V1, V2, respectively, denote strength of reflexesfor right and left triceps, with {V3, V4}, {V5, V6}, {V7, V8}, {V9, V10}similarly defined for right and left biceps, right and left wrists, right and leftknees, and right and left ankles. The correlations between variables withineach pair are large, so that the differences between variables in each pairhave small variances. This is reflected in the last five PCs, which are mainlywithin-pair contrasts, with the more highly correlated pairs correspondingto the later components.Turning to the first two PCs, there is a suggestion in the correlationmatrix that, although all correlations are positive, the variables can bedivided into two groups {V1–V6}, {V7–V10}. These correspond to sites inthe arms and legs, respectively. Reflecting this group structure, the first andsecond PCs have their largest coefficients on the first and second groups ofvariables, respectively. Because the group structure is not clear-cut, thesetwo PCs also have contributions from the less dominant group, and thefirst PC is a weighted average of variables from both groups, whereas thesecond PC is a weighted contrast between the groups.The third, fourth and fifth PCs reinforce the idea of the two groups. Thethird PC is a contrast between the two pairs of variables in the second(smaller) group and the fourth and fifth PCs both give contrasts betweenthe three pairs of variables in the first group.It is relatively rare for examples with as many as ten variables to havesuch a nicely defined structure as in the present case for all their PCs.However, as will be seen in the examples of subsequent chapters, it is notunusual to be able to deduce the structure of at least a few PCs in thismanner.
- Page 38 and 39: 1.2. A Brief History of Principal C
- Page 40 and 41: 1.2. A Brief History of Principal C
- Page 42 and 43: 2.1. Optimal Algebraic Properties o
- Page 44 and 45: 2.1. Optimal Algebraic Properties o
- Page 46 and 47: 2.1. Optimal Algebraic Properties o
- Page 48 and 49: 2.1. Optimal Algebraic Properties o
- Page 50 and 51: 2.2. Geometric Properties of Popula
- Page 52 and 53: 2.3. Principal Components Using a C
- Page 54 and 55: 2.3. Principal Components Using a C
- Page 56 and 57: 2.3. Principal Components Using a C
- Page 58 and 59: 2.4. Principal Components with Equa
- Page 60 and 61: 3Mathematical and StatisticalProper
- Page 62 and 63: where3.1. Optimal Algebraic Propert
- Page 64 and 65: 3.2. Geometric Properties of Sample
- Page 66 and 67: 3.2. Geometric Properties of Sample
- Page 68 and 69: 3.2. Geometric Properties of Sample
- Page 70 and 71: 3.3. Covariance and Correlation Mat
- Page 72 and 73: 3.3. Covariance and Correlation Mat
- Page 74 and 75: 3.4. Principal Components with Equa
- Page 76 and 77: show that X = ULA ′ .⎡ULA ′ =
- Page 78 and 79: 3.6. Probability Distributions for
- Page 80 and 81: 3.7. Inference Based on Sample Prin
- Page 82 and 83: 3.7.2 Interval Estimation3.7. Infer
- Page 84 and 85: 3.7. Inference Based on Sample Prin
- Page 86 and 87: 3.7. Inference Based on Sample Prin
- Page 90 and 91: 3.9. Models for Principal Component
- Page 92 and 93: 3.9. Models for Principal Component
- Page 94 and 95: 4Principal Components as a SmallNum
- Page 96 and 97: 4.1. Anatomical Measurements 65Tabl
- Page 98 and 99: 4.1. Anatomical Measurements 67spac
- Page 100 and 101: 4.2. The Elderly at Home 69Table 4.
- Page 102 and 103: 4.3. Spatial and Temporal Variation
- Page 104 and 105: 4.3. Spatial and Temporal Variation
- Page 106 and 107: 4.4. Properties of Chemical Compoun
- Page 108 and 109: 4.5. Stock Market Prices 77Table 4.
- Page 110 and 111: 5. Graphical Representation of Data
- Page 112 and 113: Anatomical Measurements5.1. Plottin
- Page 114 and 115: 5.1. Plotting Two or Three Principa
- Page 116 and 117: 5.2. Principal Coordinate Analysis
- Page 118 and 119: 5.2. Principal Coordinate Analysis
- Page 120 and 121: 5.2. Principal Coordinate Analysis
- Page 122 and 123: 5.3. Biplots 91columns, L is an (r
- Page 124 and 125: 5.3. Biplots 93ButandSubstituting i
- Page 126 and 127: 5.3. Biplots 95The vector gi ∗ co
- Page 128 and 129: 5.3. Biplots 97Figure 5.3. Biplot u
- Page 130 and 131: 5.3. Biplots 99Table 5.2. First two
- Page 132 and 133: 5.3. Biplots 101Figure 5.5. Biplot
- Page 134 and 135: 5.4. Correspondence Analysis 103of
- Page 136 and 137: 5.4. Correspondence Analysis 105Fig
58 3. Properties of Sample <strong>Principal</strong> <strong>Component</strong>sTable 3.4. Correlation matrix for ten variables measuring reflexes.V1 V2 V3 V4 V5 V6 V7 V8 V9 V10V1 1.00V2 0.98 1.00V3 0.60 0.62 1.00V4 0.71 0.73 0.88 1.00V5 0.55 0.57 0.61 0.68 1.00V6 0.55 0.57 0.56 0.68 0.97 1.00V7 0.38 0.40 0.48 0.53 0.33 0.33 1.00V8 0.25 0.28 0.42 0.47 0.27 0.27 0.90 1.00V9 0.22 0.21 0.19 0.23 0.16 0.19 0.40 0.41 1.00V10 0.20 0.19 0.18 0.21 0.13 0.16 0.39 0.40 0.94 1.00The correlation matrix for these data is given in Table 3.4, and thecoefficients of, and the variation accounted for by, the corresponding PCsare presented in Table 3.5. It should first be noted that the ten variablesfall into five pairs. Thus, V1, V2, respectively, denote strength of reflexesfor right and left triceps, with {V3, V4}, {V5, V6}, {V7, V8}, {V9, V10}similarly defined for right and left biceps, right and left wrists, right and leftknees, and right and left ankles. The correlations between variables withineach pair are large, so that the differences between variables in each pairhave small variances. This is reflected in the last five PCs, which are mainlywithin-pair contrasts, with the more highly correlated pairs correspondingto the later components.Turning to the first two PCs, there is a suggestion in the correlationmatrix that, although all correlations are positive, the variables can bedivided into two groups {V1–V6}, {V7–V10}. These correspond to sites inthe arms and legs, respectively. Reflecting this group structure, the first andsecond PCs have their largest coefficients on the first and second groups ofvariables, respectively. Because the group structure is not clear-cut, thesetwo PCs also have contributions from the less dominant group, and thefirst PC is a weighted average of variables from both groups, whereas thesecond PC is a weighted contrast between the groups.The third, fourth and fifth PCs reinforce the idea of the two groups. Thethird PC is a contrast between the two pairs of variables in the second(smaller) group and the fourth and fifth PCs both give contrasts betweenthe three pairs of variables in the first group.It is relatively rare for examples with as many as ten variables to havesuch a nicely defined structure as in the present case for all their PCs.However, as will be seen in the examples of subsequent chapters, it is notunusual to be able to deduce the structure of at least a few PCs in thismanner.