Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

For bidirectional RNNs, the last element of the output ISN’T the
final hidden state! Once again, since we’re using a batch_first
layer, we need to permute the hidden state’s dimensions to batchfirst
as well:
out[:, -1] == hidden.permute(1, 0, 2).view(1, -1)
Output
tensor([[ True, True, False, False]])
Bidirectional RNNs are different because the final hidden state
corresponds to the last element in the sequence for the forward
layer and to the first element in the sequence for the reverse
layer. The output, on the other hand, is aligned to sequence,
hence the difference.
Square Model
It is finally time to build a model to classify the direction in which the square was
drawn: clockwise or counterclockwise. Let’s put into practice what we’ve learned
so far and use a simple RNN to obtain the final hidden state that represents the
full sequence and use it to train a classifier layer, which is, once again, the same as
a logistic regression.
"There can be only one … hidden state."
Connor MacLeod
Data Generation
If you hadn’t noticed yet, we only have a training set. But, since our data is synthetic
anyway, let’s simply generate new data, which, by definition, wasn’t seen by the
model and therefore qualifies as validation or test data (just make sure to pick a
different seed for the generation):
Recurrent Neural Networks (RNNs) | 615