1h6QSyH

7.7 Generalized Additive Models 283
Local Linear Regression
Wage
0 50 100 200 300
Span is 0.2 (16.4 Degrees of Freedom)
Span is 0.7 (5.3 Degrees of Freedom)
20 30 40 50 60 70 80
Age
FIGURE 7.10. Local linear fits to the Wage data. The span specifies the fraction
of the data used to compute the fit at each target point.
7.7.1 GAMs for Regression Problems
A natural way to extend the multiple linear regression model
y i = β 0 + β 1 x i1 + β 2 x i2 + ···+ β p x ip + ɛ i
in order to allow for non-linear relationships between each feature and the
response is to replace each linear component β j x ij with a (smooth) nonlinear
function f j (x ij ). We would then write the model as
y i = β 0 +
p∑
f j (x ij )+ɛ i
j=1
= β 0 + f 1 (x i1 )+f 2 (x i2 )+···+ f p (x ip )+ɛ i . (7.15)
This is an example of a GAM. It is called an additive model because we
calculate a separate f j for each X j , and then add together all of their
contributions.
In Sections 7.1–7.6, we discuss many methods for fitting functions to a
single variable. The beauty of GAMs is that we can use these methods
as building blocks for fitting an additive model. In fact, for most of the
methods that we have seen so far in this chapter, this can be done fairly
trivially. Take, for example, natural splines, and consider the task of fitting
the model
wage = β 0 + f 1 (year)+f 2 (age)+f 3 (education)+ɛ (7.16)