Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
Model TrainingLet’s train our model for 100 epochs using the StepByStep class and visualize thelosses:Model Training1 n_epochs = 10023 sbs_logistic = StepByStep(4 model_logistic, binary_loss_fn, optimizer_logistic5 )6 sbs_logistic.set_loaders(train_loader, val_loader)7 sbs_logistic.train(n_epochs)fig = sbs_logistic.plot_losses()Figure 4.5 - Losses for the logistic regression modelAwful, right? It seems our model is barely learning anything! Maybe a deepermodel can do better.Deep-ish ModelThere we go, let’s add not one, but two hidden layers to our model and make itdeep-ish. We still start with a nn.Flatten layer, and the last part of our model stillis a Sigmoid, but there are two extra Linear layers before the already existingoutput layer.Deep-ish Model | 301
Let’s visualize it.Figure 4.6 - Deep-ish modelBy the way, in the figure above, the subscripts for both w and z represent the zerobasedindices for layer and unit: In the output layer, for instance, w 20 represents theweights corresponding to the first unit (#0) of the third layer (#2).What’s happening here? Let’s work out the forward pass; that is, the path frominputs (x) to output (y):1. An image is flattened to a tensor with 25 features, from x 0 to x 24 (not depictedin the figure above).2. The 25 features are forwarded to each of the five units in Hidden Layer #0.3. Each unit in Hidden Layer #0 use its weights, from w 00 to w 04 , and the featuresfrom the Input Layer to compute its corresponding outputs, from z 00 to z 04 .4. The outputs of Hidden Layer #0 are forwarded to each of the three units inHidden Layer #1 (in a way, the outputs of Hidden Layer #0 work as if they werefeatures to Hidden Layer #1).5. Each unit in Hidden Layer #1 uses its weights, from w 10 to w 12 , and the z 0 valuesfrom the preceding hidden layer to compute its corresponding outputs, from z 10to z 12 .6. The outputs of Hidden Layer #1 are forwarded to the single unit in the outputlayer (again, the outputs of Hidden Layer #1 work as if they were features to theOutput Layer).7. The unit in the Output Layer uses its weights (w 20 ) and the z 1 values from the302 | Chapter 4: Classifying Images
- Page 276 and 277: Figure 3.13 - Using a low threshold
- Page 278 and 279: Figure 3.16 - Trade-offs for two di
- Page 280 and 281: thresholds do not necessarily inclu
- Page 282 and 283: actual data, it is as bad as it can
- Page 284 and 285: If you want to learn more about bot
- Page 286 and 287: Model Training1 n_epochs = 10023 sb
- Page 288 and 289: step in your journey! What’s next
- Page 290 and 291: Chapter 4Classifying ImagesSpoilers
- Page 292 and 293: Data GenerationOur images are quite
- Page 294 and 295: Images and ChannelsIn case you’re
- Page 296 and 297: image_rgb = np.stack([image_r, imag
- Page 298 and 299: That’s fairly straightforward; we
- Page 300 and 301: • Transformations based on Tensor
- Page 302 and 303: position of an object in a picture
- Page 304 and 305: Outputtensor([[[0., 0., 0., 1., 0.]
- Page 306 and 307: Outputtensor([[[-1., -1., -1., 1.,
- Page 308 and 309: We can convert the former into the
- Page 310 and 311: composer = Compose([RandomHorizonta
- Page 312 and 313: Output<torch.utils.data.dataset.Sub
- Page 314 and 315: train_composer = Compose([RandomHor
- Page 316 and 317: The minority class should have the
- Page 318 and 319: train_loader = DataLoader(dataset=t
- Page 320 and 321: implemented in Chapter 2.1? Let’s
- Page 322 and 323: Let’s take one mini-batch of imag
- Page 324 and 325: What does our model look like? Visu
- Page 328 and 329: preceding hidden layer to compute i
- Page 330 and 331: fig = sbs_nn.plot_losses()Figure 4.
- Page 332 and 333: Equation 4.2 - Equivalence of deep
- Page 334 and 335: w_nn_equiv = w_nn_output.mm(w_nn_hi
- Page 336 and 337: Weights as PixelsDuring data prepar
- Page 338 and 339: is only 0.25 (for z = 0) and that i
- Page 340 and 341: nn.Tanh()(dummy_z)Outputtensor([-0.
- Page 342 and 343: dummy_z = torch.tensor([-3., 0., 3.
- Page 344 and 345: As you can see, in PyTorch the coef
- Page 346 and 347: Figure 4.16 - Deep model (for real)
- Page 348 and 349: Figure 4.18 - Losses (before and af
- Page 350 and 351: Equation 4.3 - Activation functions
- Page 352 and 353: Helper Function #41 def index_split
- Page 354 and 355: Model Configuration1 # Sets learnin
- Page 356 and 357: Bonus ChapterFeature SpaceThis chap
- Page 358 and 359: Affine TransformationsAn affine tra
- Page 360 and 361: Figure B.3 - Annotated model diagra
- Page 362 and 363: Figure B.5 - In the beginning…But
- Page 364 and 365: OK, now we can clearly see a differ
- Page 366 and 367: In the model above, the sigmoid fun
- Page 368 and 369: the more dimensions, the more separ
- Page 370 and 371: import randomimport numpy as npfrom
- Page 372 and 373: identity = np.array([[[[0, 0, 0],[0
- Page 374 and 375: Figure 5.4 - Striding the image, on
Model Training
Let’s train our model for 100 epochs using the StepByStep class and visualize the
losses:
Model Training
1 n_epochs = 100
2
3 sbs_logistic = StepByStep(
4 model_logistic, binary_loss_fn, optimizer_logistic
5 )
6 sbs_logistic.set_loaders(train_loader, val_loader)
7 sbs_logistic.train(n_epochs)
fig = sbs_logistic.plot_losses()
Figure 4.5 - Losses for the logistic regression model
Awful, right? It seems our model is barely learning anything! Maybe a deeper
model can do better.
Deep-ish Model
There we go, let’s add not one, but two hidden layers to our model and make it
deep-ish. We still start with a nn.Flatten layer, and the last part of our model still
is a Sigmoid, but there are two extra Linear layers before the already existing
output layer.
Deep-ish Model | 301
Change language