Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

18 2. Properties of Population Principal Components2.2 Geometric Properties of Population PrincipalComponentsIt was noted above that Property A5 can be interpreted geometrically, aswell as algebraically, and the discussion following Property A4 shows thatA4, too, has a geometric interpretation. We now look at two further, purelygeometric, properties.Property G1. Consider the family of p-dimensional ellipsoidsx ′ Σ −1 x =const. (2.2.1)The PCs define the principal axes of these ellipsoids.Proof. The PCs are defined by the transformation (2.1.1) z = A ′ x,andsince A is orthogonal, the inverse transformation is x = Az. Substitutinginto (2.2.1) gives(Az) ′ Σ −1 (Az) = const = z ′ A ′ Σ −1 Az.It is well known that the eigenvectors of Σ −1 are the same as those of Σ,and that the eigenvalues of Σ −1 are the reciprocals of those of Σ, assumingthat they are all strictly positive. It therefore follows, from a correspondingresult to (2.1.3), that AΣ −1 A = Λ −1 and hencez ′ Λ −1 z =const.This last equation can be rewrittenp∑ zk2 = const (2.2.2)λ kk=1and (2.2.2) is the equation for an ellipsoid referred to its principal axes.Equation (2.2.2) also implies that the half-lengths of the principal axes areproportional to λ 1/21 , λ 1/22 ,...,λ 1/2p . ✷This result is statistically important if the random vector x has a multivariatenormal distribution. In this case, the ellipsoids given by (2.2.1)define contours of constant probability for the distribution of x. The first(largest) principal axis of such ellipsoids will then define the direction inwhich statistical variation is greatest, which is another way of expressingthe algebraic definition of the first PC given in Section 1.1. The directionof the first PC, defining the first principal axis of constant probability ellipsoids,is illustrated in Figures 2.1 and 2.2 in Section 2.3. The secondprincipal axis maximizes statistical variation, subject to being orthogonalto the first, and so on, again corresponding to the algebraic definition. Thisinterpretation of PCs, as defining the principal axes of ellipsoids of constantdensity, was mentioned by Hotelling (1933) in his original paper.It would appear that this particular geometric property is only of directstatistical relevance if the distribution of x is multivariate normal, whereas