Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
the more dimensions, the more separable the points are, as we’ve seen in the "AreMy Data Points Separable?" section in Chapter 3.Then, by projecting it back into two dimensions, we can visualize the decisionboundary in the modified feature space. The overall shape is more complex, as if itwent through multiple foldings, as a result of the increase in dimensionality.Personally, this is one of my favorite topics, and it was the subject of my very firstblog post: "Hyper-parameters in Action! Part I—Activation Functions." [84] You canalso check out some animations I built back then for visualizing the trainingprocess using different activation functions: sigmoid, [85] hyperbolic tangent, [86] andReLU. [87]RecapAnd that’s enough for our feature space visualization journey! I hope you liked it.This is what we’ve covered:• learning what a feature space is and how a hidden layer performs affinetransformations to modify it• visualizing the effect of activation functions on the feature space• learning that the decision boundary is a straight line in the activated featurespace, but a curve in the original feature space• visualizing different decision boundaries (in original feature space) fordifferent activation functions• making a more powerful model by making it deeper• making a more powerful model by making it wider, thus increasingdimensionalityNow, let’s get back to the main track, tackling a multiclass classification problemusing convolutional neural networks (CNNs).[80] https://bit.ly/3l5XVkN[81] https://bit.ly/2QgEmYR[82] https://bit.ly/34mw0ai[83] https://bit.ly/3hgfIU6[84] https://towardsdatascience.com/hyper-parameters-in-action-a524bf5bf1c[85] https://youtu.be/4RoTHKKRXgE[86] https://youtu.be/PFNp8_V_Apg[87] https://youtu.be/Ji_05nOFLE0Recap | 343
Chapter 5ConvolutionsSpoilersIn this chapter, we will:• understand the arithmetic of convolutional layers in detail• build a model for multiclass classification• understand the role of the softmax function• use negative log-likelihood and cross-entropy losses• visualize filters learned by our convolutional neural network• understand and use hooks to capture outputs from intermediate layers• visualize feature maps to better understand what’s happening inside the modelJupyter NotebookThe Jupyter notebook corresponding to Chapter 5 [88] is part of the official DeepLearning with PyTorch Step-by-Step repository on GitHub. You can also run itdirectly in Google Colab [89] .If you’re using a local installation, open your terminal or Anaconda prompt andnavigate to the PyTorchStepByStep folder you cloned from GitHub. Then, activatethe pytorchbook environment and run jupyter notebook:$ conda activate pytorchbook(pytorchbook)$ jupyter notebookIf you’re using Jupyter’s default settings, this link should open Chapter 5’snotebook. If not, just click on Chapter05.ipynb in your Jupyter’s home page.ImportsFor the sake of organization, all libraries needed throughout the code used in anygiven chapter are imported at its very beginning. For this chapter, we’ll need thefollowing imports:344 | Chapter 5: Convolutions
- Page 318 and 319: train_loader = DataLoader(dataset=t
- Page 320 and 321: implemented in Chapter 2.1? Let’s
- Page 322 and 323: Let’s take one mini-batch of imag
- Page 324 and 325: What does our model look like? Visu
- Page 326 and 327: Model TrainingLet’s train our mod
- Page 328 and 329: preceding hidden layer to compute i
- Page 330 and 331: fig = sbs_nn.plot_losses()Figure 4.
- Page 332 and 333: Equation 4.2 - Equivalence of deep
- Page 334 and 335: w_nn_equiv = w_nn_output.mm(w_nn_hi
- Page 336 and 337: Weights as PixelsDuring data prepar
- Page 338 and 339: is only 0.25 (for z = 0) and that i
- Page 340 and 341: nn.Tanh()(dummy_z)Outputtensor([-0.
- Page 342 and 343: dummy_z = torch.tensor([-3., 0., 3.
- Page 344 and 345: As you can see, in PyTorch the coef
- Page 346 and 347: Figure 4.16 - Deep model (for real)
- Page 348 and 349: Figure 4.18 - Losses (before and af
- Page 350 and 351: Equation 4.3 - Activation functions
- Page 352 and 353: Helper Function #41 def index_split
- Page 354 and 355: Model Configuration1 # Sets learnin
- Page 356 and 357: Bonus ChapterFeature SpaceThis chap
- Page 358 and 359: Affine TransformationsAn affine tra
- Page 360 and 361: Figure B.3 - Annotated model diagra
- Page 362 and 363: Figure B.5 - In the beginning…But
- Page 364 and 365: OK, now we can clearly see a differ
- Page 366 and 367: In the model above, the sigmoid fun
- Page 370 and 371: import randomimport numpy as npfrom
- Page 372 and 373: identity = np.array([[[[0, 0, 0],[0
- Page 374 and 375: Figure 5.4 - Striding the image, on
- Page 376 and 377: Output-----------------------------
- Page 378 and 379: Outputtensor([[[[9., 5., 0., 7.],[0
- Page 380 and 381: OutputParameter containing:tensor([
- Page 382 and 383: Moreover, notice that if we were to
- Page 384 and 385: In code, as usual, PyTorch gives us
- Page 386 and 387: Outputtensor([[[[5., 5., 0., 8., 7.
- Page 388 and 389: edge = np.array([[[[0, 1, 0],[1, -4
- Page 390 and 391: A pooling kernel of two-by-two resu
- Page 392 and 393: Outputtensor([[22., 23., 11., 24.,
- Page 394 and 395: Figure 5.15 - LeNet-5 architectureS
- Page 396 and 397: • second block: produces 16-chann
- Page 398 and 399: Transformed Dataset1 class Transfor
- Page 400 and 401: LossNew problem, new loss. Since we
- Page 402 and 403: Outputtensor([4.0000, 1.0000, 0.500
- Page 404 and 405: The loss only considers the predict
- Page 406 and 407: Outputtensor([[-1.5229, -0.3146, -2
- Page 408 and 409: IMPORTANT: I can’t stress this en
- Page 410 and 411: figures at the beginning of this ch
- Page 412 and 413: The three units in the output layer
- Page 414 and 415: StepByStep Method@staticmethoddef _
- Page 416 and 417: The meow() method is totally indepe
Chapter 5
Convolutions
Spoilers
In this chapter, we will:
• understand the arithmetic of convolutional layers in detail
• build a model for multiclass classification
• understand the role of the softmax function
• use negative log-likelihood and cross-entropy losses
• visualize filters learned by our convolutional neural network
• understand and use hooks to capture outputs from intermediate layers
• visualize feature maps to better understand what’s happening inside the model
Jupyter Notebook
The Jupyter notebook corresponding to Chapter 5 [88] is part of the official Deep
Learning with PyTorch Step-by-Step repository on GitHub. You can also run it
directly in Google Colab [89] .
If you’re using a local installation, open your terminal or Anaconda prompt and
navigate to the PyTorchStepByStep folder you cloned from GitHub. Then, activate
the pytorchbook environment and run jupyter notebook:
$ conda activate pytorchbook
(pytorchbook)$ jupyter notebook
If you’re using Jupyter’s default settings, this link should open Chapter 5’s
notebook. If not, just click on Chapter05.ipynb in your Jupyter’s home page.
Imports
For the sake of organization, all libraries needed throughout the code used in any
given chapter are imported at its very beginning. For this chapter, we’ll need the
following imports:
344 | Chapter 5: Convolutions
Change language