pdfcoffee

Unsupervised Learning
# Backprop and optimize
loss = tf.reduce_mean(reconstruction_loss) + kl_div
gradients = tape.gradient(loss, model.trainable_variables)
for g in gradients:
tf.clip_by_norm(g, 15)
optimizer.apply_gradients(zip(gradients, model.trainable_
variables))
if (step + 1) % 50 == 0:
print("Epoch[{}/{}], Step [{}/{}], Reconst Loss: {:.4f},
KL Div: {:.4f}"
.format(epoch + 1, num_epochs, step + 1, num_batches,
float(reconstruction_loss), float(kl_div)))
Once the model is trained it should be able to generate images similar to the original
Fashion-MNIST images. To do so we need to use only the decoder network and we
will pass to it a randomly generated z input:
z = tf.random.normal((batch_size, latent_dim))
out = model.decode(z) # decode with sigmoid
out = tf.reshape(out, [-1, 28, 28]).numpy() * 255
out = out.astype(np.uint8)
In the following figure, you can see the result after 80 epochs; the generated images
resemble the input space:
Figure 12: Results after 80 epochs
Summary
The chapter covered the major unsupervised learning algorithms. We went
through algorithms best suited for dimension reduction, clustering, and image
reconstruction. We started with the dimension reduction algorithm PCA, then we
performed clustering using k-means and self-organized maps. After this we studied
the restricted Boltzmann machine and saw how we can use it for both dimension
reduction and image reconstruction. Next the chapter delved into stacked RBMs,
that is, deep belief networks, and we trained a DBN consisting of three RBM layers
on the MNIST dataset. Lastly, we learned about variational autoencoders, which, like
GANs, can generate images after learning the distribution of the input sample space.
[ 404 ]