Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

loss.backward().
Notebook Cell 1.5 - Autograd in action!
# Step 1 - Computes our model's predicted output - forward pass
yhat = b + w * x_train_tensor
# Step 2 - Computes the loss
# We are using ALL data points, so this is BATCH gradient
# descent. How wrong is our model? That's the error!
error = (yhat - y_train_tensor)
# It is a regression, so it computes mean squared error (MSE)
loss = (error ** 2).mean()
# Step 3 - Computes gradients for both "b" and "w" parameters
# No more manual computation of gradients!
# b_grad = 2 * error.mean()
# w_grad = 2 * (x_tensor * error).mean()
loss.backward() 1
1 New "Step 3 - Computing Gradients" using backward()
Which tensors are going to be handled by the backward() method applied to the
loss?
• b
• w
• yhat
• error
We have set requires_grad=True to both b and w, so they are obviously included in
the list. We use them both to compute yhat, so it will also make it to the list. Then
we use yhat to compute the error, which is also added to the list.
Do you see the pattern here? If a tensor in the list is used to compute another
tensor, the latter will also be included in the list. Tracking these dependencies is
exactly what the dynamic computation graph is doing, as we’ll see shortly.
What about x_train_tensor and y_train_tensor? They are involved in the
computation too, but we created them as non-gradient-requiring tensors, so
backward() does not care about them.
86 | Chapter 1: A Simple Regression Problem