pdfcoffee

Chapter 3
Simple linear regression
If we consider only one independent variable and one dependent variable, what we
get is a simple linear regression. Consider the case of house price prediction, defined
in the preceding section; the area of the house (A) is the independent variable and the
price (Y) of the house is the dependent variable. We want to find a linear relationship
between predicted price Y hat
and A, of the form:
Y hat
= A. W + b
Where b is the bias term. Thus, we need to determine W and b, such that the error
between the price Y and predicted price Y hat
is minimized. The standard method used
to estimate W and b is called the method of least squares, that is, we try to minimize
the sum of the square of errors (S). For the preceding case, the expression becomes:
NN
SS(WW, bb) = ∑ (YY ii − YY haaaa ) 2 = ∑ (YY ii − AA ii WW − bb) 2
ii=1
We want to estimate the regression coefficients, W and b, such that S is minimized.
We use the fact that the derivative of a function is 0 at its minima to get these
two equations:
NN
NN
ii=1
∂∂SS
∂∂WW = −2 ∑(YY ii − AA ii WW − bb)AA ii = 0
ii=1
NN
∂∂SS
∂∂bb = −2 ∑(YY ii − AA ii WW − bb) = 0
ii=1
These two equations can be solved to find the two unknowns. To do so, we first
expand the summation in the second equation:
NN
∑ YY ii − ∑ AA ii WW − ∑ bb = 0
ii=1
NN
ii=1
NN
ii=1
[ 89 ]