Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
Apart from returning the loss value, the inner perform_train_step_fn() functionbelow is exactly the same as the code inside the loop in Model Training V0. Thecode should look like this:Helper Function #11 def make_train_step_fn(model, loss_fn, optimizer):2 # Builds function that performs a step in the train loop3 def perform_train_step_fn(x, y):4 # Sets model to TRAIN mode5 model.train()67 # Step 1 - Computes model's predictions - forward pass8 yhat = model(x)9 # Step 2 - Computes the loss10 loss = loss_fn(yhat, y)11 # Step 3 - Computes gradients for "b" and "w" parameters12 loss.backward()13 # Step 4 - Updates parameters using gradients and14 # the learning rate15 optimizer.step()16 optimizer.zero_grad()1718 # Returns the loss19 return loss.item()2021 # Returns the function that will be called inside the22 # train loop23 return perform_train_step_fnThen we need to update our model configuration code (adding line 20 in the nextsnippet) to call this higher-order function to build a train_step_fn() function. Butwe need to run a data preparation script first.Run - Data Preparation V0%run -i data_preparation/v0.pyRethinking the Training Loop | 131
Define - Model Configuration V11 %%writefile model_configuration/v1.py23 device = 'cuda' if torch.cuda.is_available() else 'cpu'45 # Sets learning rate - this is "eta" ~ the "n"-like Greek letter6 lr = 0.178 torch.manual_seed(42)9 # Now we can create a model and send it at once to the device10 model = nn.Sequential(nn.Linear(1, 1)).to(device)1112 # Defines an SGD optimizer to update the parameters13 optimizer = optim.SGD(model.parameters(), lr=lr)1415 # Defines an MSE loss function16 loss_fn = nn.MSELoss(reduction='mean')1718 # Creates the train_step function for our model, loss function19 # and optimizer20 train_step_fn = make_train_step_fn(model, loss_fn, optimizer) 11 Creating a function that performs a training stepRun - Model Configuration V1%run -i model_configuration/v1.pyLet’s check our train_step_fn() function out!train_step_fnOutput<function __main__.make_train_step_fn.<locals>\.perform_train_step_fn(x, y)>Looking good! Now we need to update our model training to replace the codeinside the loop with a call to our newly created function.132 | Chapter 2: Rethinking the Training Loop
- Page 106 and 107: back_to_numpy = x_train_tensor.nump
- Page 108 and 109: I am assuming you’d like to use y
- Page 110 and 111: Outputtensor([0.1940], device='cuda
- Page 112 and 113: print(error.requires_grad, yhat.req
- Page 114 and 115: Output(tensor([0.], device='cuda:0'
- Page 116 and 117: 56 # need to tell it to let it go..
- Page 118 and 119: computation.If you chose "Local Ins
- Page 120 and 121: Figure 1.6 - Now parameter "b" does
- Page 122 and 123: There are many optimizers: SGD is t
- Page 124 and 125: 41 optimizer.zero_grad() 34243 prin
- Page 126 and 127: Notebook Cell 1.8 - PyTorch’s los
- Page 128 and 129: Outputarray(0.00804466, dtype=float
- Page 130 and 131: Let’s build a proper (yet simple)
- Page 132 and 133: "What do we need this for?"It turns
- Page 134 and 135: 1 Instantiating a model2 What IS th
- Page 136 and 137: In the __init__() method, we create
- Page 138 and 139: LayersA Linear model can be seen as
- Page 140 and 141: There are MANY different layers tha
- Page 142 and 143: We use magic, just like that:%run -
- Page 144 and 145: • Step 1: compute model’s predi
- Page 146 and 147: RecapFirst of all, congratulations
- Page 148 and 149: Chapter 2Rethinking the Training Lo
- Page 150 and 151: Let’s take a look at the code onc
- Page 152 and 153: Higher-Order FunctionsAlthough this
- Page 154 and 155: def exponentiation_builder(exponent
- Page 158 and 159: Our code should look like this; see
- Page 160 and 161: There is no need to load the whole
- Page 162 and 163: but if we want to get serious about
- Page 164 and 165: How does this change our code so fa
- Page 166 and 167: Run - Model Training V2%run -i mode
- Page 168 and 169: piece of code that’s going to be
- Page 170 and 171: for it. We could do the same for th
- Page 172 and 173: EvaluationHow can we evaluate the m
- Page 174 and 175: And then, we update our model confi
- Page 176 and 177: Run - Model Training V4%run -i mode
- Page 178 and 179: Loading Extension# Load the TensorB
- Page 180 and 181: browser, you’ll likely see someth
- Page 182 and 183: model’s graph (not quite the same
- Page 184 and 185: Figure 2.5 - Scalars on TensorBoard
- Page 186 and 187: Define - Model Training V51 %%write
- Page 188 and 189: If, by any chance, you ended up wit
- Page 190 and 191: The procedure is exactly the same,
- Page 192 and 193: soon, so please bear with me for no
- Page 194 and 195: After recovering our model’s stat
- Page 196 and 197: Run - Model Configuration V31 # %lo
- Page 198 and 199: This is the general structure you
- Page 200 and 201: Chapter 2.1Going ClassySpoilersIn t
- Page 202 and 203: # A completely empty (and useless)
- Page 204 and 205: # These attributes are defined here
Define - Model Configuration V1
1 %%writefile model_configuration/v1.py
2
3 device = 'cuda' if torch.cuda.is_available() else 'cpu'
4
5 # Sets learning rate - this is "eta" ~ the "n"-like Greek letter
6 lr = 0.1
7
8 torch.manual_seed(42)
9 # Now we can create a model and send it at once to the device
10 model = nn.Sequential(nn.Linear(1, 1)).to(device)
11
12 # Defines an SGD optimizer to update the parameters
13 optimizer = optim.SGD(model.parameters(), lr=lr)
14
15 # Defines an MSE loss function
16 loss_fn = nn.MSELoss(reduction='mean')
17
18 # Creates the train_step function for our model, loss function
19 # and optimizer
20 train_step_fn = make_train_step_fn(model, loss_fn, optimizer) 1
1 Creating a function that performs a training step
Run - Model Configuration V1
%run -i model_configuration/v1.py
Let’s check our train_step_fn() function out!
train_step_fn
Output
<function __main__.make_train_step_fn.<locals>\
.perform_train_step_fn(x, y)>
Looking good! Now we need to update our model training to replace the code
inside the loop with a call to our newly created function.
132 | Chapter 2: Rethinking the Training Loop
Change language