Jolliffe I. Principal Component Analysis (2ed., Springer, 2002)(518s)

288 11. Rotation and Interpretation of Principal Componentsthe selected variables in the equation. Tibshirani (1996) proposes a newmethod, the ‘least absolute shrinkage and selection operator’ or LASSO,which is a hybrid of variable selection and shrinkage estimators. The procedureshrinks the coefficients of some of the variables not simply towardszero, but exactly to zero, giving an implicit form of variable selection. TheLASSO idea can be transferred to PCA, as will now be shown.In standard multiple regression we have the equationp∑y i = β 0 + β j x ij + ɛ i , i =1, 2,...,n,j=1where y 1 ,y 2 ,...,y n are measurements on a response variable y; x ij , i =1, 2,...,n, j =1, 2,...,p, are corresponding values of p predictor variables;β 0 ,β 1 ,β 2 ,...,β p are parameters in the regression equation; and ɛ i is anerror term. In least squares regression, the parameters are estimated byminimizing the residual sum of squares,n∑ (p∑ ) 2.y i − β 0 − β j x iji=1The LASSO imposes an additional restriction on the coefficients, namelyp∑|β j |≤tj=1for some ‘tuning parameter’ t. For suitable choices of t this constrainthas the interesting property that it forces some of the coefficients in theregression equation to zero.Now consider PCA, in which linear combinations a ′ kx,k=1, 2,...,p,of the p measured variables x are found that successively have maximumvariance a ′ k Sa k, subject to a ′ k a k = 1 (and, for k ≥ 2, a ′ h a k =0,h