Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

freeze_model(alex)
The function above loops over all parameters of a given model and freezes them.
"If the model is frozen, how I am supposed to train it for my own
purpose?"
Excellent question! We have to unfreeze a small part of the model or, better yet,
replace a small part of the model. We’ll be replacing the…
Top of the Model
The "top" of the model is loosely defined as the last layer(s) of the model, usually
belonging to its classifier part. The featurizer part is usually left untouched since
we’re trying to leverage the model’s ability to generate features for us. Let’s
inspect AlexNet’s classifier once again:
print(alex.classifier)
Output
Sequential(
(0): Dropout(p=0.5, inplace=False)
(1): Linear(in_features=9216, out_features=4096, bias=True)
(2): ReLU(inplace=True)
(3): Dropout(p=0.5, inplace=False)
(4): Linear(in_features=4096, out_features=4096, bias=True)
(5): ReLU(inplace=True)
(6): Linear(in_features=4096, out_features=1000, bias=True)
)
It has two hidden layers and one output layer. The output layer produces 1,000
logits, one for each class in the ILSVRC challenge. But, unless you are playing with
the dataset used for the challenge, you’d have your own classes to compute logits
for.
In our Rock Paper Scissors dataset, we have three classes. So, we need to replace the
output layer accordingly:
510 | Chapter 7: Transfer Learning