Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Figure 10.29 - Losses—Vision Transformer
Validation losses are lower than training losses—"thank you", dropout!
Once the model is trained, we can check the embeddings of our special classifier
token:
model_vit.cls_token
Output
Parameter containing:
tensor([[[ 0.0557, -0.0345, 0.0126, -0.0300, 0.0335, -0.0422,
0.0479, -0.0248, 0.0128, -0.0605, 0.0061, -0.0178, 0.0921,
-0.0384, 0.0424, -0.0423]]], device='cuda:0', requires_grad=True)
Finally, let’s see how accurate the Vision Transformer is:
StepByStep.loader_apply(sbs_vit.val_loader, sbs_vit.correct)
Output
tensor([[76, 76],
[65, 65],
[59, 59]])
Nailed it!
860 | Chapter 10: Transform and Roll Out