Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

with no hidden layers. We’re not including the bias here, because it would make it
much harder to illustrate this point.
Show Me the Code!
If equations are not your favorite way of looking at this, let’s try using some code.
First, we need to get the weights for the layers in our deep-ish model. We can use
the weight attribute of each layer, without forgetting to detach() it from the
computation graph, so we can freely use them in other operations:
w_nn_hidden0 = model_nn.hidden0.weight.detach()
w_nn_hidden1 = model_nn.hidden1.weight.detach()
w_nn_output = model_nn.output.weight.detach()
w_nn_hidden0.shape, w_nn_hidden1.shape, w_nn_output.shape
Output
(torch.Size([5, 25]), torch.Size([3, 5]), torch.Size([1, 3]))
The shapes should match both our model’s definition and the weight matrices in
the equations above the line.
We can compute the bottom row—that is, the equivalent model—using matrix
multiplication (which happens from right to left, as in the equations):
w_nn_equiv = w_nn_output @ w_nn_hidden1 @ w_nn_hidden0
w_nn_equiv.shape
Output
torch.Size([1, 25])
"What is @ doing in the expression above?"
It is performing a matrix multiplication, exactly like torch.mm() does. We could
have written the expression above like this:
308 | Chapter 4: Classifying Images