Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Let’s take a look at the RNN’s arguments:
• input_size: It is the number of features in each data point of the sequence.
• hidden_size: It is the number of hidden dimensions you want to use.
• bias: Just like any other layer, it includes the bias in the equations.
• nonlinearity: By default, it uses the hyperbolic tangent, but you can change it
to ReLU if you want.
The four arguments above are exactly the same as those in the RNN cell. So, we can
easily create a full-fledged RNN like that:
n_features = 2
hidden_dim = 2
torch.manual_seed(19)
rnn = nn.RNN(input_size=n_features, hidden_size=hidden_dim)
rnn.state_dict()
Output
OrderedDict([('weight_ih_l0', tensor([[ 0.6627, -0.4245],
[ 0.5373, 0.2294]])),
('weight_hh_l0', tensor([[-0.4015, -0.5385],
[-0.1956, -0.6835]])),
('bias_ih_l0', tensor([0.4954, 0.6533])),
('bias_hh_l0', tensor([-0.3565, -0.2904]))])
Since the seed is exactly the same, you’ll notice that the weights and biases have
exactly the same values as our former RNN cell. The only difference is in the
parameters' names: Now they all have an _l0 suffix to indicate they belong to the
first "layer."
"What do you mean by layer? Isn’t the RNN itself a layer?"
Yes, the RNN itself can be a layer in our model. But it may have its own internal
layers! You can configure those with the following four extra arguments:
• num_layers: The RNN we’ve been using so far has one layer (the default value),
Recurrent Neural Networks (RNNs) | 603