Advanced Deep Learning with Keras

Generative Adversarial Networks (GANs)
As shown in Figure 4.1.2, a GAN is made up of two networks, a generator, and
a discriminator. The input to the generator is noise, and the output is a synthesized
signal. Meanwhile, the discriminator's input will be either a real or a synthesized
signal. Genuine signals come from the true sampled data, while the fake signals
come from the generator. All of the valid signals are labeled 1.0 (that is, 100%
probability of being real) while all the synthesized signals are labeled 0.0 (that
is, 0% probability of being real). Since the labeling process is automated, GANs
are still considered part of the unsupervised learning approach in deep learning.
The objective of the discriminator is to learn from this supplied dataset on how
to distinguish real signals from fake signals. During this part of GAN training,
only the discriminator parameters will be updated. Like a typical binary classifier,
the discriminator is trained to predict on a range of 0.0 to 1.0 in confidence values
on how close a given input signal is to the true one. However, this is only half
of the story.
At regular intervals, the generator will pretend that its output is a genuine signal
and will ask the GAN to label it as 1.0. When the fake signal is then presented
to the discriminator, naturally it will be classified as fake with a label close to 0.0.
The optimizer computes the generator parameter updates based on the presented
label (that is, 1.0). It also takes its own prediction into account when training
on this new data. In other words, the discriminator has some doubt about its
prediction, and so, GANs takes that into consideration. This time, GANs will let
the gradients backpropagate from the last layer of the discriminator down to the
first layer of the generator. However, in most practices, during this phase of training,
the discriminator parameters are temporarily frozen. The generator will use the
gradients to update its parameters and improve its ability to synthesize fake signals.
Overall, the whole process is akin to two networks competing with one another
while still cooperating at the same time. When the GAN training converges, the
end result is a generator that can synthesize signals. The discriminator thinks these
synthesized signals are real or with a label near 1.0, which means the discriminator
can then be discarded. The generator part will be useful in producing meaningful
outputs from arbitrary noise inputs.
[ 102 ]