Advanced Deep Learning with Keras

Chapter 3
As shown in Figure 3.3.2, the denoising autoencoder has practically the same
structure as the autoencoder for MNIST that we presented in the previous section.
The input is defined as:
In this formula,
x = x + noise (Equation 3.3.1)
orig
x
orig
represents the original MNIST image corrupted by noise.
The objective of the encoder is to discover how to produce the latent vector, z, that
will enable the decoder to recover x by minimizing the dissimilarity loss function
orig
such as MSE, as shown here:
i=
m
1
L( x , ) ( ) 2
orig
x ̃ = MSE = ∑ xorig − x̃
(Equation 3.3.2)
i i
m
i=
1
In this example, m is the output dimensions (for example, in MNIST m = width ×
height × channels = 28 × 28 × 1 = 784). x
orig
and
i
x̃ are the elements of x
i
orig and ̃x ,
respectively.
To implement DAE, we're going to need to make a few changes on the autoencoder
presented in the previous section. Firstly, the training input data should be corrupted
MNIST digits. The training output data is the same original clean MNIST digits. This
is like telling the autoencoder what the corrected images should be or asking it to
figure out how to remove noise given a corrupted image. Lastly, we must validate
the autoencoder on the corrupted MNIST test data.
The MNIST digit 7 shown on the left of Figure 3.3.2 is an actual corrupted image
input. The one on the right is the clean image output of a trained denoising
autoencoder.
Listing 3.3.1 shows the denoising autoencoder which has been contributed to the
Keras GitHub repository. Using the same MNIST dataset, we're able to simulate
corrupted images by adding random noise. The noise added is a Gaussian
distribution with a mean, µ = 0.5 and standard deviation of σ = 0.5 . Since adding
random noise may push the pixel data into invalid values of less than 0 or greater
than 1, the pixel values are clipped to [0.1, 1.0] range.
Everything else will remain practically the same as the autoencoder from the
previous section. We'll use the same MSE loss function and Adam optimizer as the
autoencoder. However, the number of epoch for training has increased to 10. This
is to allow sufficient parameter optimization.
[ 85 ]