Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

remained unchanged.
ResNet (MSRA Team)
The trick developed by Kaiming He, et al. was to add residual connections, or
shortcuts, to a very deep architecture.
We train neural networks with depth of over 150 layers. We propose a "deep
residual learning" framework that eases the optimization and convergence of
extremely deep networks.
Source: Results (ILSVRC2015) [120]
In a nutshell, it allows the network to more easily learn the identity function. We’ll
get back to it in the "Residual Connections" section later in this chapter. If you want
to learn more about it, the paper is called "Deep Residual Learning for Image
Recognition." [121]
By the way, Kaiming He also has an initialization scheme named
after him—sometimes referred to as "He initialization," sometimes
referred to as "Kaiming initialization"—and we’ll learn about those
in the next chapter.
Imagenette
If you are looking for a smaller, more manageable dataset that’s ImageNetlike,
Imagenette is for you! Developed by Jeremy Howard from fast.ai, it is a
subset of ten easily classified classes from ImageNet.
You can find it here: https://github.com/fastai/imagenette.
Comparing Architectures
Now that you’re familiar with some of the popular architectures (many of which are
readily available as Torchvision models), let’s compare their performance (Top-1
accuracy %), number of operations in a single forward pass (billions), and sizes (in
millions of parameters). The figure below is very illustrative in this sense.
Comparing Architectures | 503