Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Output
tensor([[ 1.0349, 0.9661],
[ 0.8055, -0.9169],
[-0.8251, -0.9499],
[-0.8670, 0.9342]])
As expected, four data points, two coordinates each. The first data point, [1.0349,
0.9661], corresponding to the top-right corner of the square, is going to be
transformed by the linear_input layers (the two red neurons):
tx = linear_input(X[0:1])
tx
Output
tensor([[0.7712, 1.4310]], grad_fn=<AddmmBackward>)
There we go: We got both t x and t h . Let’s add them together:
adding = th + tx
adding
Output
tensor([[0.4146, 1.1405]], grad_fn=<AddBackward0>)
The effect of adding t x is similar to the effect of adding the bias: It is effectively
translating the transformed hidden state to the right (by 0.7712) and up (by
1.4310).
Finally, the hyperbolic tangent activation function "compresses" the feature space
back into the (-1, 1) interval:
torch.tanh(adding)
598 | Chapter 8: Sequences