pdfcoffee

Chapter 8
batch_size, drop_remainder=True)
train_dataset = dataset.skip(test_size).batch(
batch_size, drop_remainder=True)
Our data is now ready to be used for training the seq2seq network, which we will
define next. Our encoder is an Embedding layer followed by a GRU layer. The
input to the encoder is a sequence of integers, which is converted to a sequence of
embedding vectors of size embedding_dim. This sequence of vectors is sent to an
RNN, which converts the input at each of the num_timesteps time steps to a vector
of size encoder_dim. Only the output at the last time step is returned, as shown by
the return_sequences=False.
The decoder has almost the same structure as the encoder, except that it has an
additional Dense layer that converts the vector of size decoder_dim that is output
from the RNN, into a vector that represents the probability distribution across the
target vocabulary. The decoder also returns outputs along all its time steps.
In our example network, we have chosen our embedding dimension to be 128,
followed by the encoder and decoder RNN dimension of 1024 each. Note that we
have to add 1 to the vocabulary size for both the English and French vocabularies
to account for the PAD character that was added during the pad_sequences() step:
class Encoder(tf.keras.Model):
def __init__(self, vocab_size, num_timesteps,
embedding_dim, encoder_dim, **kwargs):
super(Encoder, self).__init__(**kwargs)
self.encoder_dim = encoder_dim
self.embedding = tf.keras.layers.Embedding(
vocab_size, embedding_dim, input_length=num_timesteps)
self.rnn = tf.keras.layers.GRU(
encoder_dim, return_sequences=False, return_state=True)
def call(self, x, state):
x = self.embedding(x)
x, state = self.rnn(x, initial_state=state)
return x, state
def init_state(self, batch_size):
return tf.zeros((batch_size, self.encoder_dim))
class Decoder(tf.keras.Model):
def __init__(self, vocab_size, embedding_dim, num_timesteps,
decoder_dim, **kwargs):
super(Decoder, self).__init__(**kwargs)
self.decoder_dim = decoder_dim
[ 321 ]