Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

LRFinder
The function we’ve implemented above is fairly basic. For an implementation
with more bells and whistles, check this Python package: torch_lr_finder.
[102]
I am illustrating its usage here, which is quite similar to what we’ve done
above, but please refer to the documentation for more details.
!pip install --quiet torch-lr-finder
from torch_lr_finder import LRFinder
Instead of calling a function directly, we need to create an instance of
LRFinder first, using the typical model configuration objects (model,
optimizer, loss function, and the device). Then, we can take the range_test()
method for a spin, providing familiar arguments to it: a data loader, the upper
range for the learning rate, and the number of iterations. The reset()
method restores the original states of both model and optimizer.
torch.manual_seed(11)
new_model = CNN2(n_feature=5, p=0.3)
multi_loss_fn = nn.CrossEntropyLoss(reduction='mean')
new_optimizer = optim.Adam(new_model.parameters(), lr=3e-4)
device = 'cuda' if torch.cuda.is_available() else 'cpu'
lr_finder = LRFinder(
new_model, new_optimizer, multi_loss_fn, device=device
)
lr_finder.range_test(train_loader, end_lr=1e-1, num_iter=100)
lr_finder.plot(log_lr=True)
lr_finder.reset()
Learning Rates | 453