Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
# A completely empty (and useless) classclass StepByStep(object):passBoring, right? Let’s make it more interesting.The Constructor"From where do we start building a class?"That would be the constructor, the __init__(self) method that we’ve alreadyseen a couple of times when handling both model and dataset classes.The constructor defines the parts that make up the class. These parts are theattributes of the class. Typical attributes include:• arguments provided by the user• placeholders for other objects that are not available at the moment of creation(pretty much like delayed arguments)• variables we may want to keep track of• functions that are dynamically built using some of the arguments and higherorderfunctionsLet’s see how each of these applies to our problem.ArgumentsLet’s start with the arguments, the part that needs to be specified by the user. Atthe beginning of Chapter 2, we asked ourselves: "Would the code inside the trainingloop change if we were using a different optimizer, or loss, or even model?" The answerwas and still is, no, it wouldn’t change.So, these three elements, optimizer, loss, and model, will be our main arguments.The user needs to specify these; we can’t figure them out on our own.But there is one more piece of information needed: the device to be used fortraining the model. Instead of asking the user to supply it, we’ll automatically checkif there is a GPU available and fall back to a CPU if there isn’t. But we still want togive the user a chance to use a different device (whatever the reason may be); thus,Going Classy | 177
we add a very simple method (conveniently named to()) that allows the user tospecify a device.Our constructor (__init__()) method will initially look like this:class StepByStep(object):def __init__(self, model, loss_fn, optimizer):# Here we define the attributes of our class# We start by storing the arguments as attributes# to use laterself.model = modelself.loss_fn = loss_fnself.optimizer = optimizerself.device = 'cuda' if torch.cuda.is_available() else 'cpu'# Let's send the model to the specified device right awayself.model.to(self.device)def to(self, device):# This method allows the user to specify a different device# It sets the corresponding attribute (to be used later in# the mini-batches) and sends the model to the devicetry:self.device = deviceself.model.to(self.device)except RuntimeError:self.device = ('cuda' if torch.cuda.is_available()else 'cpu')print(f"Couldn't send it to {device}, \sending it to {self.device} instead.")self.model.to(self.device)PlaceholdersNext, let’s tackle the placeholders or delayed arguments. We expect the user toeventually provide some of these, as they are not necessarily required. There areanother three elements that fall into that category: train and validation dataloaders and a summary writer to interface with TensorBoard.We need to append the following code to the constructor method above (I am notreproducing the rest of the method here for the sake of simplicity; in the Jupyternotebook you’ll find the full code):178 | Chapter 2.1: Going Classy
- Page 152 and 153: Higher-Order FunctionsAlthough this
- Page 154 and 155: def exponentiation_builder(exponent
- Page 156 and 157: Apart from returning the loss value
- Page 158 and 159: Our code should look like this; see
- Page 160 and 161: There is no need to load the whole
- Page 162 and 163: but if we want to get serious about
- Page 164 and 165: How does this change our code so fa
- Page 166 and 167: Run - Model Training V2%run -i mode
- Page 168 and 169: piece of code that’s going to be
- Page 170 and 171: for it. We could do the same for th
- Page 172 and 173: EvaluationHow can we evaluate the m
- Page 174 and 175: And then, we update our model confi
- Page 176 and 177: Run - Model Training V4%run -i mode
- Page 178 and 179: Loading Extension# Load the TensorB
- Page 180 and 181: browser, you’ll likely see someth
- Page 182 and 183: model’s graph (not quite the same
- Page 184 and 185: Figure 2.5 - Scalars on TensorBoard
- Page 186 and 187: Define - Model Training V51 %%write
- Page 188 and 189: If, by any chance, you ended up wit
- Page 190 and 191: The procedure is exactly the same,
- Page 192 and 193: soon, so please bear with me for no
- Page 194 and 195: After recovering our model’s stat
- Page 196 and 197: Run - Model Configuration V31 # %lo
- Page 198 and 199: This is the general structure you
- Page 200 and 201: Chapter 2.1Going ClassySpoilersIn t
- Page 204 and 205: # These attributes are defined here
- Page 206 and 207: # Creates the train_step function f
- Page 208 and 209: # Builds function that performs a s
- Page 210 and 211: setattrThe setattr function sets th
- Page 212 and 213: See? We effectively modified the un
- Page 214 and 215: the random seed as arguments.This s
- Page 216 and 217: The current state of development of
- Page 218 and 219: Lossesdef plot_losses(self):fig = p
- Page 220 and 221: Run - Data Preparation V21 # %load
- Page 222 and 223: Model TrainingWe start by instantia
- Page 224 and 225: Making PredictionsLet’s make up s
- Page 226 and 227: OutputOrderedDict([('0.weight', ten
- Page 228 and 229: Run - Data Preparation V21 # %load
- Page 230 and 231: • defining our StepByStep class
- Page 232 and 233: import numpy as npimport torchimpor
- Page 234 and 235: Next, we’ll standardize the featu
- Page 236 and 237: Equation 3.1 - A linear regression
- Page 238 and 239: The odds ratio is given by the rati
- Page 240 and 241: As expected, probabilities that add
- Page 242 and 243: Sigmoid Functiondef sigmoid(z):retu
- Page 244 and 245: A picture is worth a thousand words
- Page 246 and 247: OutputOrderedDict([('linear.weight'
- Page 248 and 249: The first summation adds up the err
- Page 250 and 251: IMPORTANT: Make sure to pass the pr
we add a very simple method (conveniently named to()) that allows the user to
specify a device.
Our constructor (__init__()) method will initially look like this:
class StepByStep(object):
def __init__(self, model, loss_fn, optimizer):
# Here we define the attributes of our class
# We start by storing the arguments as attributes
# to use later
self.model = model
self.loss_fn = loss_fn
self.optimizer = optimizer
self.device = 'cuda' if torch.cuda.is_available() else 'cpu'
# Let's send the model to the specified device right away
self.model.to(self.device)
def to(self, device):
# This method allows the user to specify a different device
# It sets the corresponding attribute (to be used later in
# the mini-batches) and sends the model to the device
try:
self.device = device
self.model.to(self.device)
except RuntimeError:
self.device = ('cuda' if torch.cuda.is_available()
else 'cpu')
print(f"Couldn't send it to {device}, \
sending it to {self.device} instead.")
self.model.to(self.device)
Placeholders
Next, let’s tackle the placeholders or delayed arguments. We expect the user to
eventually provide some of these, as they are not necessarily required. There are
another three elements that fall into that category: train and validation data
loaders and a summary writer to interface with TensorBoard.
We need to append the following code to the constructor method above (I am not
reproducing the rest of the method here for the sake of simplicity; in the Jupyter
notebook you’ll find the full code):
178 | Chapter 2.1: Going Classy