Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

2Mathematical and StatisticalProperties of Population PrincipalComponentsIn this chapter many of the mathematical and statistical properties of PCsare discussed, based on a known population covariance (or correlation)matrix Σ. Further properties are included in Chapter 3 but in the contextof sample, rather than population, PCs. As well as being derived from astatistical viewpoint, PCs can be found using purely mathematical arguments;they are given by an orthogonal linear transformation of a set ofvariables optimizing a certain algebraic criterion. In fact, the PCs optimizeseveral different algebraic criteria and these optimization properties, togetherwith their statistical implications, are described in the first sectionof the chapter.In addition to the algebraic derivation given in Chapter 1, PCs can also belooked at from a geometric viewpoint. The derivation given in the originalpaper on PCA by Pearson (1901) is geometric but it is relevant to samples,rather than populations, and will therefore be deferred until Section 3.2.However, a number of other properties of population PCs are also geometricin nature and these are discussed in the second section of this chapter.The first two sections of the chapter concentrate on PCA based on acovariance matrix but the third section describes how a correlation, ratherthan a covariance, matrix may be used in the derivation of PCs. It alsodiscusses the problems associated with the choice between PCAs based oncovariance versus correlation matrices.In most of this text it is assumed that none of the variances of the PCs areequal; nor are they equal to zero. The final section of this chapter explainsbriefly what happens in the case where there is equality between some ofthe variances, or when some of the variances are zero.