pdfcoffee

Chapter 3
Multiple linear regression
The preceding example was simple, but that is rarely the case. In most problems, the
dependent variables depend upon multiple independent variables. Multiple linear
regression finds a linear relationship between the many independent input variables
(X) and the dependent output variable (Y), such that they satisfy the predicted Y
value of the form:
Y hat
= W T X + b
Where X = {x 1
, x 2
, ..., x n
} are the n independent input variables, and W = { w 1
, w 2
, ...w n
}
are the linear coefficients, with b as the bias term.
As before the linear coefficients W's are estimated using the method of least squares,
that is, minimizing the sum of squared differences between predicted values (Y hat
)
and observed values (Y). Thus, we try to minimize the loss function:
llllllll = ∑(YY ii − YY haaaaii ) 2
ii
Where the sum is over all the training samples. As you might have guessed, now
instead of two we will have n+1 equations, which we will need to simultaneously
solve. An easier alternative will be to use the TensorFlow Estimator API. We will
learn shortly how to use the TensorFlow Estimator API.
Multivariate linear regression
There can be cases where the independent variables affect more than one dependent
variable. This is the case of multivariate linear regression. Mathematically,
a multivariate regression model can be represented as:
YŶ iiii = ww 0jj + ∑ ww kkkk xx iiii
Where ii ∈ [1, … , nn] and jj ∈ [1, … , mm] . The term YŶ iiii represents the j th predicted output
value corresponding to the i th input sample, w represents the regression coefficients,
and x ik
is the k th feature of the i th input sample. The number of equations needed
to solve in this case will now be n × m. While we can solve these equations using
matrices, the process will be computationally expensive as it will involve calculating
inverse and determinants. An easier way would be to use the gradient descent
with the sum of least square error as the loss function and to use one of the many
optimizers that the TensorFlow API includes.
pp
kk=1
[ 93 ]