pdfcoffee

We train the autoencoder for 20 epochs using the following code. 20 epochs were
chosen because the MSE loss converges within this time:
num_train_steps = len(Xtrain) // BATCH_SIZE
num_test_steps = len(Xtest) // BATCH_SIZE
steps_per_epoch=num_train_steps,
epochs=NUM_EPOCHS,
validation_data=test_gen,
validation_steps=num_test_steps,
history = autoencoder.fit_generator(train_gen,
steps_per_epoch=num_train_steps,
epochs=NUM_EPOCHS,
validation_data=test_gen,
validation_steps=num_test_steps)
Chapter 9
The results of the training are shown as follows. As you can see, the training MSE
reduces from 0.1161 to 0.0824 and the validation MSE reduces from 0.1097 to 0.0820:
Since we are feeding in a matrix of embeddings, the output will also be a matrix of
word embeddings. Since the embedding space is continuous and our vocabulary is
discrete, not every output embedding will correspond to a word. The best we can
do is to find a word that is closest to the output embedding in order to reconstruct
the original text. This is a bit cumbersome, so we will evaluate our autoencoder in
a different way.
Since the objective of the autoencoder is to produce a good latent representation,
we compare the latent vectors produced from the encoder using the original input
versus the output of the autoencoder.
[ 371 ]