Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

x = torch.as_tensor(points[0:1]).float()
The RNN representing the first layer takes the sequence of data points as usual:
out0, h0 = rnn_layer0(x)
It produces the expected two outputs: a sequence of hidden states (out0) and the
final hidden state (h0) for this layer.
Next, it uses the sequence of hidden states as inputs for the next layer:
out1, h1 = rnn_layer1(out0)
The second layer produces the expected two outputs again: another sequence of
hidden states (out1) and the final hidden state (h1) for this layer.
The overall output of the stacked RNN must have two elements as well:
• A sequence of hidden states, produced by the last layer (out1).
• The concatenation of final hidden states of all layers.
out1, torch.cat([h0, h1])
Output
(tensor([[[-0.7533, -0.7711],
[-0.0566, -0.5960],
[ 0.4324, -0.2908],
[ 0.1563, -0.5152]]], grad_fn=<TransposeBackward1>),
tensor([[[-0.5297, 0.3551]],
[[ 0.1563, -0.5152]]], grad_fn=<CatBackward>))
Done! We’ve replicated the inner workings of a stacked RNN using two simple
RNNs. You can double-check the results by feeding the sequence of data points to
the actual stacked RNN itself:
610 | Chapter 8: Sequences