Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Output
OrderedDict([('linear.weight', tensor([[0.5406, 0.5869]])),
('linear.bias', tensor([-0.1657]))])
Did you notice that state_dict() contains parameters from the linear layer only?
Even though the model has a second sigmoid layer, this layer does not contain any
parameters since it does not need to learn anything: The sigmoid function will be
the same regardless of which model it is a part of.
Loss
We already have a model, and now we need to define an appropriate loss for it. A
binary classification problem calls for the binary cross-entropy (BCE) loss,
sometimes known as log loss.
The BCE loss requires the predicted probabilities, as returned by the sigmoid
function, and the true labels (y) for its computation. For each data point i in the
training set, it starts by computing the error corresponding to the point’s true
class.
If the data point belongs to the positive class (y=1), we would like our model to
predict a probability close to one, right? A perfect one would result in the
logarithm of one, which is zero. It makes sense; a perfect prediction means zero
loss. It goes like this:
Equation 3.11 - Error for a data point in the positive class
What if the data point belongs to the negative class (y=0)? Then we cannot simply
use the predicted probability. Why not? Because the model outputs the probability
of a point’s belonging to the positive, not the negative, class. Luckily, the latter can
be easily computed:
Equation 3.12 - Probability of a data point’s belonging to the negative class
And thus, the error associated with a data point’s belonging to the negative class
Loss | 221