Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

354 13. Principal Component Analysis for Special Types of DataObservations of the atmosphere certainly do not constitute a designedexperiment, but a technique proposed by Zheng et al. (2001) is includedhere because of its connections to analysis of variance. Measurements ofmeteorological fields can be thought of as the sum of long-term variabilitycaused by external forcing and the slowly varying internal dynamics of theatmosphere, and short-term day-to-day weather variability. The first termis potentially predictable over seasonal or longer time scales, whereas thesecond term is not. It is therefore of interest to separate out the potentiallypredictable component and examine its major sources of variation. Zhenget al. (2001) do this by estimating the covariance matrix of the day-to-dayvariation and subtracting it from the ‘overall’ covariance matrix, whichassumes independence of short- and long-term variation. A PCA is thendone on the resulting estimate of the covariance matrix for the potentiallypredictable variation, in order to find potentially predictable patterns.Returning to designed experiments, in optimal design it is desirable toknow the effect of changing a design by deleting design points or augmentingit with additional design points. Jensen (1998) advocates the use ofprincipal components of the covariance matrix of predicted values at a chosenset of design points to investigate the effects of such augmentation ordeletion. He calls these components principal predictors, though they arequite different from the entities with the same name defined by Thacker(1999) and discussed in Section 9.3.3. Jensen (1998) illustrates the use ofhis principal predictors for a variety of designs.13.5 Common Principal Components andComparisons of Principal ComponentsSuppose that observations on a p-variate random vector x may have comefrom any one of G distinct populations, and that the mean and covariancematrix for the gth population are, respectively, µ g , Σ g , g =1, 2,...,G.Thisis the situation found in discriminant analysis (see Section 9.1) althoughin discriminant analysis it is often assumed that all the Σ g are the same,so that the populations differ only in their means. If the Σ g are all thesame, then the ‘within-population PCs’ are the same for all G populations,though, as pointed out in Section 9.1, within-population PCs are oftendifferent from PCs found by pooling data from all populations together.If the Σ g are different, then there is no uniquely defined set of withinpopulationPCs that is common to all populations. However, a number ofauthors have examined ‘common principal components,’ which can usefullybe defined in some circumstances where the Σ g are not all equal. The ideaof ‘common’ PCs arises if we suspect that the same components underliethe covariance matrices of each group, but that they have different weightsin different groups. For example, if anatomical measurements are made on