Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

I guess it is time to feed the full sequence to the RNN cell, right? You may be
tempted to do it like this:
# WRONG!
rnn_cell(X)
Output
tensor([[ 0.3924, 0.8146],
[ 0.7864, 0.5266],
[-0.0047, -0.2897],
[-0.6817, 0.1109]], grad_fn=<TanhBackward>)
This is wrong! Remember, the RNN cell has two inputs: one hidden state and one
data point.
"Where is the hidden state then?"
That’s exactly the problem! If not provided, it defaults to the zeros corresponding
to the initial hidden state. So, the call above is not processing four steps of a
sequence, but rather processing the first step of what it is assuming to be four
sequences.
To effectively use the RNN cell in a sequence, we need to loop over the data points
and provide the updated hidden state at each step:
hidden = torch.zeros(1, hidden_dim)
for i in range(X.shape[0]):
out = rnn_cell(X[i:i+1], hidden)
print(out)
hidden = out
Output
tensor([[0.3924, 0.8146]], grad_fn=<TanhBackward>)
tensor([[ 0.4347, -0.0481]], grad_fn=<TanhBackward>)
tensor([[-0.1521, -0.3367]], grad_fn=<TanhBackward>)
tensor([[-0.5297, 0.3551]], grad_fn=<TanhBackward>)
600 | Chapter 8: Sequences