Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Loss
New problem, new loss. Since we’re embracing multiclass classification now, we
need to use a different loss. And, once again, it all starts with our "favorite" subject:
logits.
Logits
In binary classification problems, the model would produce one logit, and one logit
only, for each data point. It makes sense, as binary classification is about answering
a simple question: "Does a given data point belong to the positive class?"
The logit output represented the log odds ratio (remember that, from Chapter 3?)
of answering "yes" to the question above. The log odds ratio of a "no" answer was
simply the inverse. There was no need to pose any other question to make a
decision. And we used a sigmoid function to map logits to probabilities. It was a
simple world :-)
But a multiclass classification is more complex: We need to ask more questions;
that is, we need to get log odds ratios for every possible class. In other words, we
need as many logits as there are classes.
"But a sigmoid takes only one logit. I guess we need something else to
get probabilities, right?"
Absolutely correct! The function we’re looking for here is called softmax.
Softmax
The softmax function returns, for each class, the contribution that a given class
had to the sum of odds ratios. The class with a higher odds ratio will have the
largest contribution and thus the highest probability.
A Multiclass Classification Problem | 375