Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Output
tensor([[-5.4936e-02, -8.3816e-05]], grad_fn=<MulBackward0>)
The LSTM cell must return both states, hidden and cell, in that order, as a tuple:
(h_prime, c_prime)
Output
(tensor([[-5.4936e-02, -8.3816e-05]], grad_fn=<MulBackward0>),
tensor([[-0.1340, -0.0004]], grad_fn=<AddBackward0>))
That’s it! Wasn’t that bad, right? The formulation of the LSTM may seem scary at
first sight, especially if you bump into a huge sequence of equations using all
weights and biases at once, but it doesn’t have to be that way.
Finally, let’s take a quick sanity check, feeding the same input to the original LSTM
cell:
lstm_cell(first_corner)
Output
(tensor([[-5.4936e-02, -8.3816e-05]], grad_fn=<MulBackward0>),
tensor([[-0.1340, -0.0004]], grad_fn=<AddBackward0>))
And we’re done with cells. I guess you know what comes next…
LSTM Layer
The nn.LSTM layer takes care of the hidden and cell states handling for us, no
matter how long the input sequence is. We’ve been through this once with the RNN
layer and then again with the GRU layer. The arguments, inputs, and outputs of the
LSTM are almost exactly the same as those for the GRU, except for the fact that, as
you already know, LSTMs return two states (hidden and cell) with the same shape
instead of one. By the way, you can create stacked LSTMs and bidirectional LSTMs
too.
Long Short-Term Memory (LSTM) | 649