Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
Share Embed Flag
Download ePaper

Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

peiying410632
from peiying410632 More from this publisher
22.02.2024 Views

Equation 4.2 - Equivalence of deep and shallow modelsThe first row below the line shows the sequence of matrices. The bottom rowshows the result of the matrix multiplication. This result is exactly the sameoperation shown in the "Notation" subsection of the shallow model; that is, thelogistic regression.In a nutshell, a model with any number of hidden layers has an equivalent modelDeep-ish Model | 307

with no hidden layers. We’re not including the bias here, because it would make itmuch harder to illustrate this point.Show Me the Code!If equations are not your favorite way of looking at this, let’s try using some code.First, we need to get the weights for the layers in our deep-ish model. We can usethe weight attribute of each layer, without forgetting to detach() it from thecomputation graph, so we can freely use them in other operations:w_nn_hidden0 = model_nn.hidden0.weight.detach()w_nn_hidden1 = model_nn.hidden1.weight.detach()w_nn_output = model_nn.output.weight.detach()w_nn_hidden0.shape, w_nn_hidden1.shape, w_nn_output.shapeOutput(torch.Size([5, 25]), torch.Size([3, 5]), torch.Size([1, 3]))The shapes should match both our model’s definition and the weight matrices inthe equations above the line.We can compute the bottom row—that is, the equivalent model—using matrixmultiplication (which happens from right to left, as in the equations):w_nn_equiv = w_nn_output @ w_nn_hidden1 @ w_nn_hidden0w_nn_equiv.shapeOutputtorch.Size([1, 25])"What is @ doing in the expression above?"It is performing a matrix multiplication, exactly like torch.mm() does. We couldhave written the expression above like this:308 | Chapter 4: Classifying Images

Equation 4.2 - Equivalence of deep and shallow models

The first row below the line shows the sequence of matrices. The bottom row

shows the result of the matrix multiplication. This result is exactly the same

operation shown in the "Notation" subsection of the shallow model; that is, the

logistic regression.

In a nutshell, a model with any number of hidden layers has an equivalent model

Deep-ish Model | 307

logo

products

Resources

Company

Terms of service
Privacy policy
Cookie policy
Cookie settings
Imprint
Change language
Made with love in Switzerland
© 2024 Yumpu.com all rights reserved

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!