pdfcoffee

Recurrent Neural Networks
self.embedding = tf.keras.layers.Embedding(
vocab_size, embedding_dim, input_length=num_timesteps)
self.rnn = tf.keras.layers.GRU(
decoder_dim, return_sequences=True, return_state=True)
self.dense = tf.keras.layers.Dense(vocab_size)
def call(self, x, state):
x = self.embedding(x)
x, state = self.rnn(x, state)
x = self.dense(x)
return x, state
embedding_dim = 256
encoder_dim, decoder_dim = 1024, 1024
encoder = Encoder(vocab_size_en+1,
embedding_dim, maxlen_en, encoder_dim)
decoder = Decoder(vocab_size_fr+1,
embedding_dim, maxlen_fr, decoder_dim)
Now that we have defined our Encoder and Decoder classes, let us revisit the
dimensions of their inputs and outputs. The following piece of (throwaway) code can
be used to print out the dimensions of the various inputs and outputs of the system.
It has been left in for convenience as a commented out block in the code supplied
with this chapter:
for encoder_in, decoder_in, decoder_out in train_dataset:
encoder_state = encoder.init_state(batch_size)
encoder_out, encoder_state = encoder(encoder_in, encoder_state)
decoder_state = encoder_state
decoder_pred, decoder_state = decoder(decoder_in, decoder_state)
break
print("encoder input
:", encoder_in.shape)
print("encoder output
:", encoder_out.shape, "state:",
encoder_state.shape)
print("decoder output (logits):", decoder_pred.shape, "state:",
decoder_state.shape)
print("decoder output (labels):", decoder_out.shape)
This produces the following output, which is in line with our expectations. The
encoder input is a batch of a sequence of integers, each sequence being of size 8,
which is the maximum number of tokens in our English sentences, so its dimension
is (batch_size, maxlen_en).
[ 322 ]