Advanced Deep Learning with Keras

Generative Adversarial Networks (GANs)
This can then be rewritten more aptly as a value function:
( ) ( ) ( )
( θ , θ ) =− ( θ , θ )
( G ) ( G ) ( D )
D G D
V L (Equation 4.1.3)
From the perspective of the generator, Equation 4.1.3 should be minimized. From
the point of view of the discriminator, the value function should be maximized.
Therefore, the generator training criterion can be written as a minimax problem:
( G)
( D) ( G) ( D)
( G) ( D)
θ θ
( )
θ ∗ = arg minmin V θ , θ
(Equation 4.1.4)
Occasionally, we'll try to fool the discriminator by pretending that the synthetic
( D)
data is real with label 1.0. By maximizing with respect to θ , the optimizer sends
gradient updates to the discriminator parameters to consider this synthetic data as
real. At the same time, by minimizing with respect to
( G)
θ , the optimizer will train
the generator's parameters on how to trick the discriminator. However, in practice,
the discriminator is confident in its prediction in classifying the synthetic data as fake
and will not update its parameters. Furthermore, the gradient updates are small and
have diminished significantly as they propagate to the generator layers. As a result,
the generator fails to converge:
Figure 4.1.4: Training the generator is like training a network using a binary cross-entropy loss function.
The fake data from the generator is presented as genuine.
The solution is to reformulate the loss function of the generator in the form:
( θ , θ ) =−E z
log ( z)
( G ) ( G ) ( D )
( )
L D G (Equation 4.1.5)
[ 104 ]