Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

50 3. Properties of Sample Principal Componentsdescriptive tool, although Mandel (1972) argued that retaining m PCs inan analysis implicitly assumes a model for the data, based on (3.5.3). Therehas recently been an upsurge of interest in models related to PCA; this isdiscussed further in Section 3.9.Although the purely inferential side of PCA is a very small part of theoverall picture, the ideas of inference can sometimes be useful and arediscussed briefly in the next three subsections.3.7.1 Point EstimationThe maximum likelihood estimator (MLE) for Σ, the covariance matrix ofa multivariate normal distribution, is not S, but (n−1)nS (see, for example,Press (1972, Section 7.1) for a derivation). This result is hardly surprising,given the corresponding result for the univariate normal. If λ, l, α k , a k andrelated quantities are defined as in the previous section, then the MLEs ofλ and α k ,k=1, 2,...,p, can be derived from the MLE of Σ and are equalto ˆλ = (n−1)nl,and ˆα k = a k ,k=1, 2,...,p, assuming that the elements ofλ are all positive and distinct. The MLEs are the same in this case as theestimators derived by the method of moments. The MLE for λ k is biasedbut asymptotically unbiased, as is the MLE for Σ. As noted in the previoussection, l itself, as well as ˆλ, is a biased estimator for λ, but ‘corrections’can be made to reduce the bias.In the case where some of the λ k are equal, the MLE for their commonvalue is simply the average of the corresponding l k , multiplied by (n−1)/n.The MLEs of the α k corresponding to equal λ k are not unique; the (p × q)matrix whose columns are MLEs of α k corresponding to equal λ k can bemultiplied by any (q × q) orthogonal matrix, where q is the multiplicity ofthe eigenvalues, to get another set of MLEs.Most often, point estimates of λ, α k are simply given by l, a k , and theyare rarely accompanied by standard errors. An exception is Flury (1997,Section 8.6). Jackson (1991, Sections 5.3, 7.5) goes further and gives examplesthat not only include estimated standard errors, but also estimatesof the correlations between elements of l and between elements of a k anda k ′. The practical implications of these (sometimes large) correlations arediscussed in Jackson’s examples. Flury (1988, Sections 2.5, 2.6) gives athorough discussion of asymptotic inference for functions of the variancesand coefficients of covariance-based PCs.If multivariate normality cannot be assumed, and if there is no obviousalternative distributional assumption, then it may be desirable to use a‘robust’ approach to the estimation of the PCs: this topic is discussed inSection 10.4.