Advanced Deep Learning with Keras

Generative Adversarial Networks (GANs)
else:
strides = 1
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Conv2DTranspose(filters=filters,
kernel_size=kernel_size,
strides=strides,
padding='same')(x)
x = Activation('sigmoid')(x)
generator = Model(inputs, x, name='generator')
return generator
The discriminator is similar to many CNN-based classifiers. The input is a 28 × 28 × 1
MNIST image that is classified as either real (1.0) or fake (0.0). There are four CNN
layers. Except for the last convolution, each Conv2D uses strides = 2 to down
sample the feature maps by two. Each Conv2D is then preceded by a Leaky ReLU
layer. The final filter size is 256, while the initial filter size is 32 and doubles every
convolution layer. The final filter size of 128 also works. However, we'll find that
the generated images look better with 256. The final output layer is flattened, and
a single unit Dense layer generates the prediction between 0.0 to 1.0 after scaling
by the sigmoid activation layer. The output is modeled as a Bernoulli distribution.
Hence, the binary cross-entropy loss function is used.
After building the generator and discriminator models, the adversarial model is
made by concatenating the generator and discriminator networks. Both discriminator
and adversarial networks use the RMSprop optimizer. The learning rate for the
discriminator is 2e-4 while for the adversarial network, it is 1e-4. RMSprop decay
rates of 6e-8 for discriminator and 3e-8 for the adversarial network are applied.
Setting the learning rate of the adversarial equal to half of the discriminator
will result in a more stable training. We'll recall from Figure 4.1.3 and 4.1.4, that
the GAN training has two parts: discriminator training and generator training,
which is adversarial training, with discriminator weights frozen.
Listing 4.2.2 shows the implementation of the discriminator in Keras. A function
is defined to build the discriminator model. In Listing 4.2.3, we'll illustrate how to
build GAN models. Firstly, the discriminator model is built and following on from
that the generator model is instantiated. The adversarial model is just the generator
and the discriminator put together. Across many GANs, the batch size of 64 appears
to be the most common. The network parameters are shown in Listing 4.2.3.
[ 108 ]