Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
"What do we need this for?"It turns out, state dictionaries can also be used for checkpointing a model, as we willsee in Chapter 2.DeviceIMPORTANT: We need to send our model to the same devicewhere the data is. If our data is made of GPU tensors, our modelmust "live" inside the GPU as well.If we were to send our dummy model to a device, it would look like this:torch.manual_seed(42)# Creates a "dummy" instance of our ManualLinearRegression model# and sends it to the devicedummy = ManualLinearRegression().to(device)Forward PassThe forward pass is the moment when the model makes predictions.Remember: You should make predictions calling model(x).DO NOT call model.forward(x)!Otherwise, your model’s hooks will not work (if you have them).We can use all these handy methods to change our code, which should be lookinglike this:Model | 107
Notebook Cell 1.10 - PyTorch’s model in action: no more manual prediction / forward step!1 # Sets learning rate - this is "eta" ~ the "n"-like2 # Greek letter3 lr = 0.145 # Step 0 - Initializes parameters "b" and "w" randomly6 torch.manual_seed(42)7 # Now we can create a model and send it at once to the device8 model = ManualLinearRegression().to(device) 1910 # Defines an SGD optimizer to update the parameters11 # (now retrieved directly from the model)12 optimizer = optim.SGD(model.parameters(), lr=lr)1314 # Defines an MSE loss function15 loss_fn = nn.MSELoss(reduction='mean')1617 # Defines number of epochs18 n_epochs = 10001920 for epoch in range(n_epochs):21 model.train() # What is this?!? 22223 # Step 1 - Computes model's predicted output - forward pass24 # No more manual prediction!25 yhat = model(x_train_tensor) 32627 # Step 2 - Computes the loss28 loss = loss_fn(yhat, y_train_tensor)2930 # Step 3 - Computes gradients for both "b" and "w" parameters31 loss.backward()3233 # Step 4 - Updates parameters using gradients and34 # the learning rate35 optimizer.step()36 optimizer.zero_grad()3738 # We can also inspect its parameters using its state_dict39 print(model.state_dict())108 | Chapter 1: A Simple Regression Problem
- Page 82 and 83: computing the loss, as shown in the
- Page 84 and 85: • visualizing the effects of usin
- Page 86 and 87: If you’re using Jupyter’s defau
- Page 88 and 89: Notebook Cell 1.1 - Splitting synth
- Page 90 and 91: Step 2# Step 2 - Computing the loss
- Page 92 and 93: Output[0.49671415] [-0.1382643][0.8
- Page 94 and 95: Notebook Cell 1.2 - Implementing gr
- Page 96 and 97: # Sanity Check: do we get the same
- Page 98 and 99: Outputtensor(3.1416)tensor([1, 2, 3
- Page 100 and 101: Outputtensor([[1., 2., 1.],[1., 1.,
- Page 102 and 103: dummy_array = np.array([1, 2, 3])du
- Page 104 and 105: n_cudas = torch.cuda.device_count()
- Page 106 and 107: back_to_numpy = x_train_tensor.nump
- Page 108 and 109: I am assuming you’d like to use y
- Page 110 and 111: Outputtensor([0.1940], device='cuda
- Page 112 and 113: print(error.requires_grad, yhat.req
- Page 114 and 115: Output(tensor([0.], device='cuda:0'
- Page 116 and 117: 56 # need to tell it to let it go..
- Page 118 and 119: computation.If you chose "Local Ins
- Page 120 and 121: Figure 1.6 - Now parameter "b" does
- Page 122 and 123: There are many optimizers: SGD is t
- Page 124 and 125: 41 optimizer.zero_grad() 34243 prin
- Page 126 and 127: Notebook Cell 1.8 - PyTorch’s los
- Page 128 and 129: Outputarray(0.00804466, dtype=float
- Page 130 and 131: Let’s build a proper (yet simple)
- Page 134 and 135: 1 Instantiating a model2 What IS th
- Page 136 and 137: In the __init__() method, we create
- Page 138 and 139: LayersA Linear model can be seen as
- Page 140 and 141: There are MANY different layers tha
- Page 142 and 143: We use magic, just like that:%run -
- Page 144 and 145: • Step 1: compute model’s predi
- Page 146 and 147: RecapFirst of all, congratulations
- Page 148 and 149: Chapter 2Rethinking the Training Lo
- Page 150 and 151: Let’s take a look at the code onc
- Page 152 and 153: Higher-Order FunctionsAlthough this
- Page 154 and 155: def exponentiation_builder(exponent
- Page 156 and 157: Apart from returning the loss value
- Page 158 and 159: Our code should look like this; see
- Page 160 and 161: There is no need to load the whole
- Page 162 and 163: but if we want to get serious about
- Page 164 and 165: How does this change our code so fa
- Page 166 and 167: Run - Model Training V2%run -i mode
- Page 168 and 169: piece of code that’s going to be
- Page 170 and 171: for it. We could do the same for th
- Page 172 and 173: EvaluationHow can we evaluate the m
- Page 174 and 175: And then, we update our model confi
- Page 176 and 177: Run - Model Training V4%run -i mode
- Page 178 and 179: Loading Extension# Load the TensorB
- Page 180 and 181: browser, you’ll likely see someth
"What do we need this for?"
It turns out, state dictionaries can also be used for checkpointing a model, as we will
see in Chapter 2.
Device
IMPORTANT: We need to send our model to the same device
where the data is. If our data is made of GPU tensors, our model
must "live" inside the GPU as well.
If we were to send our dummy model to a device, it would look like this:
torch.manual_seed(42)
# Creates a "dummy" instance of our ManualLinearRegression model
# and sends it to the device
dummy = ManualLinearRegression().to(device)
Forward Pass
The forward pass is the moment when the model makes predictions.
Remember: You should make predictions calling model(x).
DO NOT call model.forward(x)!
Otherwise, your model’s hooks will not work (if you have them).
We can use all these handy methods to change our code, which should be looking
like this:
Model | 107
Change language