Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Chapter 10
Transform and Roll Out
Spoilers
In this chapter, we will:
• modify the multi-headed attention mechanism to use narrow attention
• use layer normalization to standardize individual data points
• stack "layers" together to build Transformer encoders and decoders
• add layer normalization, dropout, and residual connections to each "sublayer"
operation
• learn the difference between norm-last and norm-first "sub-layers"
• train a Transformer to predict a target sequence from a source sequence
• build and train a Vision Transformer to perform image classification
Jupyter Notebook
The Jupyter notebook corresponding to Chapter 10 [146] is part of the official Deep
Learning with PyTorch Step-by-Step repository on GitHub. You can also run it
directly in Google Colab [147] .
If you’re using a local installation, open your terminal or Anaconda prompt and
navigate to the PyTorchStepByStep folder you cloned from GitHub. Then, activate
the pytorchbook environment and run jupyter notebook:
$ conda activate pytorchbook
(pytorchbook)$ jupyter notebook
If you’re using Jupyter’s default settings, this link should open Chapter 10’s
notebook. If not, just click on Chapter10.ipynb in your Jupyter’s home page.
Imports
For the sake of organization, all libraries needed throughout the code used in any
given chapter are imported at its very beginning. For this chapter, we’ll need the
796 | Chapter 10: Transform and Roll Out