Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

h_prime = n*(1-z) + initial_hidden*z
h_prime
Output
tensor([[-0.5635, -0.1470]], grad_fn=<AddBackward0>)
Now, let’s take a quick sanity check, feeding the same input to the original GRU
cell:
gru_cell(first_corner)
Output
tensor([[-0.5635, -0.1470]], grad_fn=<AddBackward0>)
Perfect match!
But, then again, you’re likely not inclined to loop over the sequence yourself while
using a GRU cell, right? You probably want to use a full-fledged…
GRU Layer
The nn.GRU layer takes care of the hidden state handling for us, no matter how long
the input sequence is. We’ve been through this once with the RNN layer. The
arguments, inputs, and outputs are almost exactly the same for both of them,
except for one small difference: You cannot choose a different activation function
anymore. That’s it.
And yes, you can create stacked GRUs and bidirectional GRUs as well. The logic
doesn’t change a bit—the only difference is that you’ll be using a fancier GRU cell
instead of the basic RNN cell.
So, let’s go straight to creating a model using a gated recurring unit.
634 | Chapter 8: Sequences