1h6QSyH

280 7. Moving Beyond Linearity
Smoothing Spline
Wage
0 50 100 200 300
16 Degrees of Freedom
6.8 Degrees of Freedom (LOOCV)
20 30 40 50 60 70 80
Age
FIGURE 7.8. Smoothing spline fits to the Wage data. The red curve results
from specifying 16 effective degrees of freedom. For the blue curve, λ was found
automatically by leave-one-out cross-validation, which resulted in 6.8 effective
degrees of freedom.
Figure 7.8 shows the results from fitting a smoothing spline to the Wage
data. The red curve indicates the fit obtained from pre-specifying that we
would like a smoothing spline with 16 effective degrees of freedom. The blue
curve is the smoothing spline obtained when λ is chosen using LOOCV; in
this case, the value of λ chosen results in 6.8 effective degrees of freedom
(computed using (7.13)). For this data, there is little discernible difference
between the two smoothing splines, beyond the fact that the one with 16
degrees of freedom seems slightly wigglier. Since there is little difference
between the two fits, the smoothing spline fit with 6.8 degrees of freedom
is preferable, since in general simpler models are better unless the data
provides evidence in support of a more complex model.
7.6 Local Regression
Local regression is a different approach for fitting flexible non-linear func- local
regression
tions, which involves computing the fit at a target point x 0 using only the
nearby training observations. Figure 7.9 illustrates the idea on some simulated
data, with one target point near 0.4, and another near the boundary
at 0.05. In this figure the blue line represents the function f(x) fromwhich
the data were generated, and the light orange line corresponds to the local
regression estimate ˆf(x). Local regression is described in Algorithm 7.1.
Note that in Step 3 of Algorithm 7.1, the weights K i0 will differ for each
value of x 0 . In other words, in order to obtain the local regression fit at a
new point, we need to fit a new weighted least squares regression model by