Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
Notebook Cell 1.8 - PyTorch’s loss in action: no more manual loss computation!1 # Sets learning rate - this is "eta" ~ the "n"-like2 # Greek letter3 lr = 0.145 # Step 0 - Initializes parameters "b" and "w" randomly6 torch.manual_seed(42)7 b = torch.randn(1, requires_grad=True, \8 dtype=torch.float, device=device)9 w = torch.randn(1, requires_grad=True, \10 dtype=torch.float, device=device)1112 # Defines an SGD optimizer to update the parameters13 optimizer = optim.SGD([b, w], lr=lr)1415 # Defines an MSE loss function16 loss_fn = nn.MSELoss(reduction='mean') 11718 # Defines number of epochs19 n_epochs = 10002021 for epoch in range(n_epochs):22 # Step 1 - Computes model's predicted output - forward pass23 yhat = b + w * x_train_tensor2425 # Step 2 - Computes the loss26 # No more manual loss!27 # error = (yhat - y_train_tensor)28 # loss = (error ** 2).mean()29 loss = loss_fn(yhat, y_train_tensor) 23031 # Step 3 - Computes gradients for both "b" and "w" parameters32 loss.backward()3334 # Step 4 - Updates parameters using gradients and35 # the learning rate36 optimizer.step()37 optimizer.zero_grad()3839 print(b, w)Loss | 101
1 Defining a loss function2 New "Step 2 - Computing Loss" using loss_fn()Outputtensor([1.0235], device='cuda:0', requires_grad=True)tensor([1.9690], device='cuda:0', requires_grad=True)Let’s take a look at the loss value at the end of training…lossOutputtensor(0.0080, device='cuda:0', grad_fn=
- Page 76 and 77: Let’s look at the cross-sections
- Page 78 and 79: Zero Mean and Unit Standard Deviati
- Page 80 and 81: Sure, in the real world, you’ll n
- Page 82 and 83: computing the loss, as shown in the
- Page 84 and 85: • visualizing the effects of usin
- Page 86 and 87: If you’re using Jupyter’s defau
- Page 88 and 89: Notebook Cell 1.1 - Splitting synth
- Page 90 and 91: Step 2# Step 2 - Computing the loss
- Page 92 and 93: Output[0.49671415] [-0.1382643][0.8
- Page 94 and 95: Notebook Cell 1.2 - Implementing gr
- Page 96 and 97: # Sanity Check: do we get the same
- Page 98 and 99: Outputtensor(3.1416)tensor([1, 2, 3
- Page 100 and 101: Outputtensor([[1., 2., 1.],[1., 1.,
- Page 102 and 103: dummy_array = np.array([1, 2, 3])du
- Page 104 and 105: n_cudas = torch.cuda.device_count()
- Page 106 and 107: back_to_numpy = x_train_tensor.nump
- Page 108 and 109: I am assuming you’d like to use y
- Page 110 and 111: Outputtensor([0.1940], device='cuda
- Page 112 and 113: print(error.requires_grad, yhat.req
- Page 114 and 115: Output(tensor([0.], device='cuda:0'
- Page 116 and 117: 56 # need to tell it to let it go..
- Page 118 and 119: computation.If you chose "Local Ins
- Page 120 and 121: Figure 1.6 - Now parameter "b" does
- Page 122 and 123: There are many optimizers: SGD is t
- Page 124 and 125: 41 optimizer.zero_grad() 34243 prin
- Page 128 and 129: Outputarray(0.00804466, dtype=float
- Page 130 and 131: Let’s build a proper (yet simple)
- Page 132 and 133: "What do we need this for?"It turns
- Page 134 and 135: 1 Instantiating a model2 What IS th
- Page 136 and 137: In the __init__() method, we create
- Page 138 and 139: LayersA Linear model can be seen as
- Page 140 and 141: There are MANY different layers tha
- Page 142 and 143: We use magic, just like that:%run -
- Page 144 and 145: • Step 1: compute model’s predi
- Page 146 and 147: RecapFirst of all, congratulations
- Page 148 and 149: Chapter 2Rethinking the Training Lo
- Page 150 and 151: Let’s take a look at the code onc
- Page 152 and 153: Higher-Order FunctionsAlthough this
- Page 154 and 155: def exponentiation_builder(exponent
- Page 156 and 157: Apart from returning the loss value
- Page 158 and 159: Our code should look like this; see
- Page 160 and 161: There is no need to load the whole
- Page 162 and 163: but if we want to get serious about
- Page 164 and 165: How does this change our code so fa
- Page 166 and 167: Run - Model Training V2%run -i mode
- Page 168 and 169: piece of code that’s going to be
- Page 170 and 171: for it. We could do the same for th
- Page 172 and 173: EvaluationHow can we evaluate the m
- Page 174 and 175: And then, we update our model confi
1 Defining a loss function
2 New "Step 2 - Computing Loss" using loss_fn()
Output
tensor([1.0235], device='cuda:0', requires_grad=True)
tensor([1.9690], device='cuda:0', requires_grad=True)
Let’s take a look at the loss value at the end of training…
loss
Output
tensor(0.0080, device='cuda:0', grad_fn=<MeanBackward0>)
What if we wanted to have it as a Numpy array? I guess we could just use numpy()
again, right? (And cpu() as well, since our loss is in the cuda device.)
loss.cpu().numpy()
Output
RuntimeError
Traceback (most recent call last)
<ipython-input-43-58c76a7bac74> in <module>
----> 1 loss.cpu().numpy()
RuntimeError: Can't call numpy() on Variable that requires
grad. Use var.detach().numpy() instead.
What happened here? Unlike our data tensors, the loss tensor is actually computing
gradients; to use numpy(), we need to detach() the tensor from the computation
graph first:
loss.detach().cpu().numpy()
102 | Chapter 1: A Simple Regression Problem
Change language