Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

14.5. Three-Mode, Multiway and Multiple Group PCA 397derlying factors will very often be correlated, and that it is too restrictiveto force them to be uncorrelated, let alone independent (see, for exampleCattell (1978, p. 128); Richman (1986)).14.5 Three-Mode, Multiway and Multiple GroupPrincipal Component AnalysisPrincipal component analysis is usually done on a single (n×p) data matrixX, but there are extensions to many other data types. In this section wediscuss briefly the case where there are additional ‘modes’ in the data. Aswell as rows (individuals) and columns (variables) there are other layers,such as different time periods.The ideas for three-mode methods were first published by Tucker in themid-1960s (see, for example, Tucker, 1966) and by the early 1980s the topicof three-mode principal component analysis was, on its own, the subject ofa 398-page book (Kroonenberg, 1983a). A 33-page annotated bibliography(Kroonenberg, 1983b) gave a comprehensive list of references for theslightly wider topic of three-mode factor analysis. The term ‘three-mode’refers to data sets that have three modes by which the data may be classified.For example, when PCs are obtained for several groups of individualsas in Section 13.5, there are three modes corresponding to variables, groupsand individuals. Alternatively, we might have n individuals, p variables andt time points, so that ‘individuals,’ ‘variables’ and ‘time points’ define thethree modes. In this particular case we have effectively n time series of pvariables, or a single time series of np variables. However, the time pointsneed not be equally spaced, nor is the time-order of the t repetitions necessarilyrelevant in the sort of data for which three-mode PCA is used, in thesame way that neither individuals nor variables usually have any particulara priori ordering.Let x ijk be the observed value of the jth variable for the ith individualmeasured on the kth occasion. The basic idea in three-mode analysis is toapproximate x ijk by the modelm∑ q∑ s∑˜x ijk =a ih b jl c kr g hlr .h=1 l=1 r=1The values m, q, s are less, and if possible very much less, than n, p,t, respectively, and the parameters a ih ,b jl, c kr ,g hlr ,i= 1, 2,...,n, h =1, 2,...,m, j =1, 2,...,p, l =1, 2,...,q, k =1, 2,...,t, r =1, 2,...,sare chosen to give a good fit of ˜x ijk to x ijk for all i, j, k. There are anumber of methods for solving this problem and, like ordinary PCA, theyinvolve finding eigenvalues and eigenvectors of cross-product or covariancematrices, in this case by combining two of the modes (for example, combine