Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

14.1. Additive Principal Components and Principal Curves 381Since Kramer’s paper appeared, a number of authors in the neural networkliterature have noted limitations to his procedure and have suggestedalternatives or modifications (see, for example, Jia et al., 2000), althoughit is now used in a range of disciplines including climatology (Monahan,2001). Dong and McAvoy (1996) propose an algorithm that combines theprincipal curves of Section 14.1.2 (Hastie and Stuetzle, 1989) with the autoassociativeneural network set-up of Kramer (1991). Principal curves alonedo not allow the calculation of ‘scores’ with respect to the curves for newobservations, but their combination with a neural network enables suchquantities to be computed.An alternative approach, based on a so-called input-training net, is suggestedby Tan and Mavrovouniotis (1995). In such networks, the inputs arenot fixed, but are trained along with the other parameters of the network.With a single input the results of the algorithm are equivalent to principalcurves, but with a larger number of inputs there is increased flexibility togo beyond the additive model underlying principal curves.Jia et al. (2000) use Tan and Mavrovouniotis’s (1995) input-trainingnet, but have an ordinary linear PCA as a preliminary step. The nonlinearalgorithm is then conducted on the first m linear PCs, where mis chosen to be sufficiently large, ensuring that only PCs with very smallvariances are excluded. Jia and coworkers suggest that around 97% of thetotal variance should be retained to avoid discarding dimensions that mightinclude important non-linear variation. The non-linear components are usedin process control (see Section 13.7), and in an example they give improvedfault detection compared to linear PCs (Jia et al., 2000). The preliminarystep reduces the dimensionality of the data from 37 variables to 12 linearPCs, whilst retaining 98% of the variation.Kambhatla and Leen (1997) introduce non-linearity in a different way, usinga piecewise-linear or ‘local’ approach. The p-dimensional space definedby the possible values of x is partitioned into Q regions, and linear PCsare then found separately for each region. Kambhatla and Leen (1997) notethat this local PCA provides a faster algorithm than a global non-linearneural network. A clustering algorithm is used to define the Q regions.Roweis and Saul (2000) describe a locally linear embedding algorithm thatalso generates local linear reconstructions of observations, this time basedon a set of ‘neighbours’ of each observation. Tarpey (2000) implements asimilar but more restricted idea. He looks separately at the first PCs withintwo regions defined by the sign of the first PC for the whole data set, as ameans of determining the presence of non-linear structure in the data.14.1.4 Other Aspects of Non-LinearityWe saw in Section 5.3 that biplots can provide an informative way of displayingthe results of a PCA. Modifications of these ‘classical’ biplots tobecome non-linear are discussed in detail by Gower and Hand (1996, Chap-