Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
Figure 5.22 - Feature maps (classifier)The hidden layer performed an affine transformation (remember those?), reducingthe dimensionality from sixteen to ten dimensions. Next, the activation function, aReLU, eliminated negative values, resulting in the "activated" feature space in themiddle row.Finally, the output layer used these ten values to compute three logits, one foreach class. Even without transforming them into probabilities, we know that thelargest logit wins. The largest logit is shown as the brightest pixel, so we can tellwhich class was predicted by looking at the three shades of gray and picking theindex of the brightest one.The classifier got eight out of ten right. It made wrong predictions for images #6and #8. Unsurprisingly, these are the two images that got their vertical linessuppressed. The filter doesn’t seem to work so well whenever the vertical line istoo close to the left edge of the image."How good is the model actually?"Good question! Let’s check it out.AccuracyIn Chapter 3, we made predictions using our own predict() method and usedScikit-Learn’s metrics module to evaluate them. Now, let’s build a method that alsotakes features (x) and labels (y), as returned by a data loader, and that takes allnecessary steps to produce two values for each class: the number of correctpredictions and the number of data points in that class.Visualizing Filters and More! | 407
StepByStep Methoddef correct(self, x, y, threshold=.5):self.model.eval()yhat = self.model(x.to(self.device))y = y.to(self.device)self.model.train()# We get the size of the batch and the number of classes# (only 1, if it is binary)n_samples, n_dims = yhat.shapeif n_dims > 1:# In a multiclass classification, the largest logit# always wins, so we don't bother getting probabilities# This is PyTorch's version of argmax,# but it returns a tuple: (max value, index of max value)_, predicted = torch.max(yhat, 1)else:n_dims += 1# In binary classification, we NEED to check if the# last layer is a sigmoid (and then it produces probs)if isinstance(self.model, nn.Sequential) and \isinstance(self.model[-1], nn.Sigmoid):predicted = (yhat > threshold).long()# or something else (logits), which we need to convert# using a sigmoidelse:predicted = (torch.sigmoid(yhat) > threshold).long()# How many samples got classifiedcorrectly for each classresult = []for c in range(n_dims):n_class = (y == c).sum().item()n_correct = (predicted[y == c] == c).sum().item()result.append((n_correct, n_class))return torch.tensor(result)setattr(StepByStep, 'correct', correct)If the labels have two or more columns, it means we’re dealing with a multiclass408 | Chapter 5: Convolutions
- Page 382 and 383: Moreover, notice that if we were to
- Page 384 and 385: In code, as usual, PyTorch gives us
- Page 386 and 387: Outputtensor([[[[5., 5., 0., 8., 7.
- Page 388 and 389: edge = np.array([[[[0, 1, 0],[1, -4
- Page 390 and 391: A pooling kernel of two-by-two resu
- Page 392 and 393: Outputtensor([[22., 23., 11., 24.,
- Page 394 and 395: Figure 5.15 - LeNet-5 architectureS
- Page 396 and 397: • second block: produces 16-chann
- Page 398 and 399: Transformed Dataset1 class Transfor
- Page 400 and 401: LossNew problem, new loss. Since we
- Page 402 and 403: Outputtensor([4.0000, 1.0000, 0.500
- Page 404 and 405: The loss only considers the predict
- Page 406 and 407: Outputtensor([[-1.5229, -0.3146, -2
- Page 408 and 409: IMPORTANT: I can’t stress this en
- Page 410 and 411: figures at the beginning of this ch
- Page 412 and 413: The three units in the output layer
- Page 414 and 415: StepByStep Method@staticmethoddef _
- Page 416 and 417: The meow() method is totally indepe
- Page 418 and 419: StepByStep Methoddef visualize_filt
- Page 420 and 421: dummy_model = nn.Linear(1, 1)dummy_
- Page 422 and 423: dummy_listOutput[(Linear(in_feature
- Page 424 and 425: Output{Conv2d(1, 1, kernel_size=(3,
- Page 426 and 427: will be the externally defined vari
- Page 428 and 429: Removing Hookssbs_cnn1.remove_hooks
- Page 430 and 431: return figsetattr(StepByStep, 'visu
- Page 434 and 435: classification: The predicted class
- Page 436 and 437: convolutional layers to our model a
- Page 438 and 439: Capturing Outputsfeaturizer_layers
- Page 440 and 441: the filters learned by the model pr
- Page 442 and 443: given chapter are imported at its v
- Page 444 and 445: Data PreparationThe data preparatio
- Page 446 and 447: model anyway. We’ll use it to com
- Page 448 and 449: StepByStep Method@staticmethoddef m
- Page 450 and 451: "What’s wrong with the colors?"Th
- Page 452 and 453: three_channel_filter = np.array([[[
- Page 454 and 455: Fancier Model (Constructor)class CN
- Page 456 and 457: Fancier Model (Classifier)def class
- Page 458 and 459: torch.manual_seed(44)dropping_model
- Page 460 and 461: Outputtensor([0.1000, 0.2000, 0.300
- Page 462 and 463: Figure 6.8 - Output distribution fo
- Page 464 and 465: Adaptive moment estimation (Adam) u
- Page 466 and 467: torch.manual_seed(13)# Model Config
- Page 468 and 469: Outputtorch.Size([5, 3, 3, 3])Its s
- Page 470 and 471: Choosing a learning rate that works
- Page 472 and 473: Higher-Order Learning Rate Function
- Page 474 and 475: Perfect! Now let’s build the actu
- Page 476 and 477: ax.set_xlabel('Learning Rate')ax.se
- Page 478 and 479: LRFinderThe function we’ve implem
- Page 480 and 481: value in our moving average has an
StepByStep Method
def correct(self, x, y, threshold=.5):
self.model.eval()
yhat = self.model(x.to(self.device))
y = y.to(self.device)
self.model.train()
# We get the size of the batch and the number of classes
# (only 1, if it is binary)
n_samples, n_dims = yhat.shape
if n_dims > 1:
# In a multiclass classification, the largest logit
# always wins, so we don't bother getting probabilities
# This is PyTorch's version of argmax,
# but it returns a tuple: (max value, index of max value)
_, predicted = torch.max(yhat, 1)
else:
n_dims += 1
# In binary classification, we NEED to check if the
# last layer is a sigmoid (and then it produces probs)
if isinstance(self.model, nn.Sequential) and \
isinstance(self.model[-1], nn.Sigmoid):
predicted = (yhat > threshold).long()
# or something else (logits), which we need to convert
# using a sigmoid
else:
predicted = (torch.sigmoid(yhat) > threshold).long()
# How many samples got classified
correctly for each class
result = []
for c in range(n_dims):
n_class = (y == c).sum().item()
n_correct = (predicted[y == c] == c).sum().item()
result.append((n_correct, n_class))
return torch.tensor(result)
setattr(StepByStep, 'correct', correct)
If the labels have two or more columns, it means we’re dealing with a multiclass
408 | Chapter 5: Convolutions