pdfcoffee

Chapter 10
And the reconstructed images:
out = rbm.rbm_reconstruct(test_data)
# Plotting original and reconstructed images
row, col = 2, 8
idx = np.random.randint(0, 100, row * col // 2)
f, axarr = plt.subplots(row, col, sharex=True, sharey=True,
figsize=(20,4))
for fig, row in zip([test_data,out], axarr):
for i,ax in zip(idx,row):
ax.imshow(tf.reshape(fig[i],[28, 28]), cmap='Greys_r')
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
Figure 9: Image reconstruction using an RBM
What do you think? Are RBMs better than Autoencoders? Try training the RBM on
noisy input and see how good it is with reconstructing noisy images.
Deep belief networks
Now that we have a good understanding of RBMs and know how to train them
using contrastive divergence, we can move toward the first successful deep neural
network architecture, the deep belief networks (DBNs). Proposed in 2006 in the
paper by Hinton and his team in the paper A fast learning algorithm for deep belief nets.
Before this model it was very difficult to train deep architectures, not just because
of the limited computing resources, but also, as discussed in Chapter 9, Autoencoders,
because of the vanishing gradient problem. In DBNs it was first demonstrated how
deep architectures can be trained via greedy layer-wise training.
In the simplest terms, DBNs are just stacked RBMs. Each RBM is trained separately
using the contrastive divergence. We start with the training of the first RBM layer.
Once it is trained, we train the second RBM layer. The visible units of the second
RBM are now fed the output of the hidden units of the first RBM, when it is fed the
input data. The procedure is repeated with each RBM layer addition.
[ 397 ]