Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

328 12. PCA for Time Series and Other Non-Independent Data12.4 PCA and Non-Independent Data—SomeAdditional TopicsIn this section we collect together a number of topics from time series,and other contexts in which non-independent data arise, where PCA orrelated techniques are used. Section 12.4.1 describes PCA in the frequencydomain, and Section 12.4.2 covers growth curves and longitudinal data. InSection 12.4.3 a summary is given of another idea (optimal fingerprints)from climatology, though one that is rather different in character fromthose presented in Section 12.2. Section 12.4.4 discusses spatial data, andthe final subsection provides brief coverage of a few other topics, includingnon-independence induced by survey designs.12.4.1 PCA in the Frequency DomainThe idea of PCA in the frequency domain clearly has no counterpart fordata sets consisting of independent observations. Brillinger (1981, Chapter9) devotes a whole chapter to the subject (see also Priestley et al.(1974)). To see how frequency domain PCs are derived, note that PCs fora p-variate random vector x, with zero mean, can be obtained by finding(p × q) matrices B, C such thatE[(x − Cz) ′ (x − Cz)]is minimized, where z = B ′ x. This is equivalent to the criterion that definesProperty A5 in Section 2.1 It turns out that B = C and that the columnsof B are the first q eigenvectors of Σ, the covariance matrix of x, so thatthe elements of z are the first q PCs for x. This argument can be extendedto a time series of p variables as follows. Suppose that our series is ...x −1 ,x 0 , x 1 , x 2 ,... and that E[x t ]=0 for all t. Define∞∑z t = B ′ t−ux u ,u=−∞and estimate x t by ∑ ∞u=−∞ C t−uz u , where...B t−1 , B t , B t+1 , B t+2 ,...,C t−1 , C t , C t+1 , C t+2 ,...are (p × q) matrices that minimize[( ∞∑ ) ∗E x t − C t−u z u(x t −u=−∞∞∑u=−∞C t−u z u)],where * denotes conjugate transpose. The difference between this formulationand that for ordinary PCs above is that the relationships between zand x are in terms of all values of x t and z t , at different times, rather thanbetween a single x and z. Also, the derivation is in terms of general complex