Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

14.1. Additive Principal Components and Principal Curves 377in some way to make the optimization problem tractable. One choice is touse step functions, which leads back towards Gifi’s (1990) system of nonlinearPCA. Besse and Ferraty (1995) favour an approach based on splines.They contrast their proposal, in which flexibility of the functional transformationis controlled by the choice of smoothing parameters, with earlierspline-based procedures controlled by the number and positioning of knots(see, for example, van Rijckevorsel (1988) and Winsberg (1988)). Usingsplines as Besse and Ferraty do is equivalent to adding a roughness penaltyfunction to the quantity to be minimized. This is similar to Besse et al.’s(1997) approach to analysing functional data described in Section 12.3.4using equation (12.3.6).As with Gifi’s (1990) non-linear PCA, Besse and Ferraty’s (1995) proposalis implemented by means of an alternating least squares algorithmand, as in Besse and de Falgerolles (1993) for the linear case (see Section6.1.5), bootstrapping of residuals from a q-dimensional model is usedto decide on the best fit. Here, instead of simply using the bootstrap tochoose q, simultaneous optimization with respect q and with respect to thesmoothing parameters which determine the function f(x) is needed. At thisstage it might be asked ‘where is the PCA in all this?’ The name ‘PCA’is still appropriate because the q-dimensional subspace is determined byan optimal set of q linear functions of the vector of transformed randomvariables f(x), and it is these linear functions that are the non-linear PCs.14.1.2 Additive Principal Components and Principal CurvesFowlkes and Kettenring (1985) note that one possible objective for transformingdata before performing a PCA is to find near-singularities in thetransformed data. In other words, x ′ =(x 1 ,x 2 ,...,x p ) is transformed tof ′ (x) =(f 1 (x 1 ),f 2 (x 2 ),...,f p (x p )), and we are interested in finding linearfunctions a ′ f(x) off(x) for which var[a ′ f(x)] ≈ 0. Fowlkes and Kettenring(1985) suggest looking for a transformation that minimizes the determinantof the correlation matrix of the transformed variables. The last fewPCs derived from this correlation matrix should then identify the requirednear-constant relationships, if any exist.A similar idea underlies additive principal components, which are discussedin detail by Donnell et al. (1994). The additive principal componentstake the form ∑ pj=1 φ j(x j ) instead of ∑ pj=1 a jx j in standard PCA, and, aswith Fowlkes and Kettenring (1985), interest centres on components forwhich var[ ∑ pj=1 φ j(x j )] is small. To define a non-linear analogue of PCAthere is a choice of either an algebraic definition that minimizes variance,or a geometric definition that optimizes expected squared distance fromthe additive manifold ∑ pj=1 φ j(x j ) = const. Once we move away from linearPCA, the two definitions lead to different solutions, and Donnell etal. (1994) choose to minimize variance. The optimization problem to be