Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

48 3. Properties of Sample Principal ComponentsLet l k , a k ,fork =1, 2,...,p be the eigenvalues and eigenvectors of S,respectively, and let λ k , α k ,fork =1, 2,...,p, be the eigenvalues andeigenvectors of Σ, respectively. Also, let l, λ be the p-element vectors consistingof the l k and λ k , respectively and let the jth elements of a k , α k bea kj , α kj , respectively. [The notation a jk was used for the jth element of a kin the previous section, but it seems more natural to use a kj in this and thenext, section. We also revert to using l k to denote the kth eigenvalue of Srather than that of X ′ X.] The best known and simplest results concerningthe distribution of the l k and the a k assume, usually quite realistically, thatλ 1 >λ 2 > ··· >λ p > 0; in other words all the population eigenvalues arepositive and distinct. Then the following results hold asymptotically:(i) all of the l k are independent of all of the a k ;(ii) l and the a k are jointly normally distributed;(iii)(iv)E(l) =λ, E(a k )=α k , k =1, 2,...,p; (3.6.1)⎧⎨ 2λ 2 kk = kcov(l k ,l k ′)=′ ,n − 1⎩0 k ≠ k ′ ,cov(a kj ,a k ′ j ′)= ⎧⎪ ⎨⎪ ⎩λ k(n − 1)p∑l=1l≠kλ l α lj α lj ′(λ l − λ k ) 2 k = k ′ ,(3.6.2)(3.6.3)− λ kλ k ′α kj α k ′ j ′(n − 1)(λ k − λ k ′) 2 k ≠ k ′ .An extension of the above results to the case where some of the λ k maybe equal to each other, though still positive, is given by Anderson (1963),and an alternative proof to that of Anderson can be found in Srivastavaand Khatri (1979, Section 9.4.1).It should be stressed that the above results are asymptotic and thereforeonly approximate for finite samples. Exact results are available, butonly for a few special cases, such as when Σ = I (Srivastava and Khatri,1979, p. 86) and more generally for l 1 , l p , the largest and smallest eigenvalues(Srivastava and Khatri, 1979, p. 205). In addition, better but morecomplicated approximations can be found to the distributions of l and thea k in the general case (see Srivastava and Khatri, 1979, Section 9.4; Jackson,1991, Sections 4.2, 4.5; and the references cited in these sources). Onespecific point regarding the better approximations is that E(l 1 ) >λ 1 andE(l p )