Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

And indeed, it is—it is structurally the same as our mini-batch function (Helper
Function #2). So let’s use it once again!
There is just one small, yet important detail to consider: Remember no_grad()?
We used it in Chapter 1 to avoid messing with PyTorch’s dynamic computation
graph during the (manual) update of the parameters. And it is making a comeback
now—we need to use it to wrap our new validation’s inner loop:
torch.no_grad(): Even though it won’t make a difference in our
simple model, it is a good practice to wrap the validation inner
loop with this context manager [59] to disable any gradient
computation that you may inadvertently trigger—gradients
belong in training, not in validation steps.
Now, our training loop should look like this:
Define - Model Training V4
1 %%writefile model_training/v4.py
2
3 # Defines number of epochs
4 n_epochs = 200
5
6 losses = []
7 val_losses = [] 3
8
9 for epoch in range(n_epochs):
10 # inner loop
11 loss = mini_batch(device, train_loader, train_step_fn)
12 losses.append(loss)
13
14 # VALIDATION - no gradients in validation!
15 with torch.no_grad(): 1
16 val_loss = mini_batch(device, val_loader, val_step_fn)2
17 val_losses.append(val_loss) 3
1 Using no_grad() as context manager to prevent gradient computation
2 Performing a validation step
3 Keeping track of validation loss
150 | Chapter 2: Rethinking the Training Loop