Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Classification Losses Showdown!
Honestly, I always feel this whole thing is a bit confusing, especially for someone
who’s learning it for the first time.
Which loss functions take logits as inputs? Should I add a (log)softmax layer or not?
Can I use the weight argument to handle imbalanced datasets? Too many
questions, right?
So, here is a table to help you figure out the landscape of loss functions for
classification problems, both binary and multiclass:
BCE Loss
BCE With
NLL Loss
Cross-Entropy
Logits Loss
Loss
Classification binary binary multiclass multiclass
Input (each
data point)
probability logit array of log
probabilities
array of logits
Label (each
float (0.0 or
float (0.0 or
long (class
long (class
data point)
1.0)
1.0)
index)
index)
Model’s last
layer
Sigmoid - LogSoftmax -
weight
not class
not class
class weights
class weights
argument
weights
weights
pos_weight
argument
n/a "weighted" loss n/a n/a
Model Configuration
Let’s build our first convolutional neural network for real! We can use the typical
convolutional block: convolutional layer, activation function, pooling layer. Our
images are quite small, so we only need one of those.
We still need to decide how many channels our convolutional layer is going to
produce. In general, the number of channels increases with each convolutional
block. For the sake of simplicity (and later visualization), let’s keep a single channel.
We also need to decide on a kernel size (the receptive field or gray regions in the
384 | Chapter 5: Convolutions