pdfcoffee

Unsupervised Learning
Now that we have the basic architecture of VAEs, the question arises of how they can
be trained, since the maximum likelihood of the training data and posterior density
are intractable? The network is trained by maximizing the lower bound of the log
data likelihood. Thus, the loss term consists of two components: generation loss,
which is obtained from the decoder network through sampling, and the Kullback–
Leibler (KL) divergence term, also called the latent loss.
Generation loss ensures that the image generated by the decoder and the image used
to train the network are similar, and latent loss ensures that the posterior distribution
q(z|x) is close to the prior ppΘ(zz) . Since the encoder uses Gaussian distribution
for sampling, the latent loss measures how closely the latent variables match this
distribution.
Once the VAE is trained, we can use only the decoder network to generate new
images. Let us try coding a VAE. This time we are using the Fashion-MNIST dataset;
you learned about this dataset in Chapter 5, Advanced Convolutional Neural Networks.
The dataset contains Zalando's (https://github.com/zalandoresearch/fashionmnist)
article images. The test-train split is exactly the same as for MNIST, that is,
60,000 train images and 10,000 test images. The size of each image is also 28 × 28, so
you can easily replace the codes running on the MNIST dataset with the Fashion-
MNIST dataset. The code in this section has been adapted from https://github.
com/dragen1860/TensorFlow-2.x-Tutorials. As the first step we, as usual,
import all the necessary libraries:
import tensorflow as tf
import numpy as np
from matplotlib import pyplot as plt
Let us fix the seeds for random number, so that the results are reproducible. We can
also add an assert statement to ensure that our code runs on TensorFlow 2.0 or above:
np.random.seed(333)
tf.random.set_seed(333)
assert tf.__version__.startswith('2.'), "TensorFlow Version Below 2.0"
Before going ahead with making the VAE, let us also explore the Fashion-MNIST
dataset a little. The dataset is available in the TensorFlow Keras API:
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.fashion_
mnist.load_data()
x_train, x_test = x_train.astype(np.float32)/255., x_test.astype(np.
float32)/255.
print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)
--------------------------------------------------
[ 400 ]