Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

2 1. IntroductionFigure 1.1. Plot of 50 observations on two variables x 1,x 2.Although PCA does not ignore covariances and correlations, it concentrateson variances. The first step is to look for a linear function α ′ 1x ofthe elements of x having maximum variance, where α 1 is a vector of pconstants α 11 , α 12 ,...,α 1p ,and ′ denotes transpose, so thatα ′ 1x = α 11 x 1 + α 12 x 2 + ···+ α 1p x p =p∑α 1j x j .Next, look for a linear function α ′ 2x, uncorrelated with α ′ 1x having maximumvariance, and so on, so that at the kth stage a linear function α ′ k xis found that has maximum variance subject to being uncorrelated withα ′ 1x, α ′ 2x,...,α ′ k−1 x.Thekth derived variable, α′ kx is the kth PC. Up top PCs could be found, but it is hoped, in general, that most of the variationin x will be accounted for by m PCs, where m ≪ p. The reductionin complexity achieved by transforming the original variables to PCs willbe demonstrated in many examples later in the book, but it will be usefulhere to consider first the unrealistic, but simple, case where p =2.Theadvantage of p = 2 is, of course, that the data can be plotted exactly intwo dimensions.j=1