Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

2.1. Optimal Algebraic Properties of Population Principal Components 15and, from (2.1.10),Σ − Σ xz Σ −1zz Σ zx =p∑k=(q+1)λ k α k α ′ k.Finding a linear function of x having maximum conditional variancereduces to finding the eigenvalues and eigenvectors of the conditional covariancematrix, and it easy to verify that these are simply (λ (q+1) , α (q+1) ),(λ (q+2) , α (q+2) ),...,(λ p , α p ). The eigenvector associated with the largestof these eigenvalues is α (q+1) , so the required linear function is α ′ (q+1) x,namely the (q + 1)th PC.Property A4. As in Properties A1, A2, consider the transformationy = B ′ x.Ifdet(Σ y ) denotes the determinant of the covariance matrix y,then det(Σ y ) is maximized when B = A q .Proof. Consider any integer, k, between 1 and q, and let S k =the subspace of p-dimensional vectors orthogonal to α 1 ,...,α k−1 . Thendim(S k )=p − k + 1, where dim(S k ) denotes the dimension of S k .Thektheigenvalue, λ k ,ofΣ satisfiesλ k = Supα∈S kα≠0{ α ′ }Σαα ′ .αSuppose that µ 1 >µ 2 > ··· >µ q , are the eigenvalues of B ′ ΣB and thatγ 1 , γ 2 , ··· , γ q , are the corresponding eigenvectors. Let T k = the subspaceof q-dimensional vectors orthogonal to γ k+1 , ··· , γ q , with dim(T k )=k.Then, for any non-zero vector γ in T k ,γ ′ B ′ ΣBγγ ′ γ≥ µ k .Consider the subspace ˜S k of p-dimensional vectors of the form Bγ for γ inT k .dim( ˜S k )=dim(T k )=k(because B is one-to-one; in fact,B preserves lengths of vectors).From a general result concerning dimensions of two vector spaces, we haveButsodim(S k ∩ ˜S k )+dim(S k + ˜S k )=dimS k +dim˜S k .dim(S k + ˜S k ) ≤ p, dim(S k )=p − k + 1 and dim( ˜S k )=k,dim(S k ∩ ˜S k ) ≥ 1.