Advanced Deep Learning with Keras

Chapter 4
Figure 4.1.3: Training the discriminator is similar to training a binary classifier network using binary
cross-entropy loss. The fake data is supplied by the generator while real data is from true samples.
As shown in the preceding figure, the discriminator can be trained by minimizing
the loss function in the following equation:
( θ , θ ) =−E
x~
p
log ( x) −Ez
log( 1−
( ( z)
))
( D) ( G) ( D)
L D D G (Equation 4.1.1)
data
The equation is just the standard binary cross-entropy cost function. The loss is
the negative sum of the expectation of correctly identifying real data, D ( x)
, and
the expectation of 1.0 minus correctly identifying synthetic data, 1−D( G ( z)
). The
log does not change the location of the local minima. Two mini-batches of data are
supplied to the discriminator during training:
1. x , real from sampled data (that is, x ~ p data
) with label 1.0
x′ = G z , fake data from the generator with label 0.0
2. ( )
In order to minimize the loss function, the discriminator parameters, θ , will be
updated through backpropagation by correctly identifying the genuine data, D( x)
, and synthetic data, 1−D( G ( z)
). Correctly identifying real data is equivalent
to D ( x) →1.0
while correctly classifying fake data is the same as D( G ( z)
) → 0.0
or ( 1−D( G ( z)
))
→1.0
. In this equation, z is the arbitrary encoding or noise
vector that is used by the generator to synthesize new signals. Both contribute
to minimizing the loss function.
To train the generator, GAN considers the total of the discriminator and generator
losses as a zero-sum game. The generator loss function is simply the negative of the
discriminator loss function:
( ) ( ) ( )
( θ , θ ) =− ( θ , θ )
( G ) ( G ) ( D )
D G D
L L (Equation 4.1.2)
[ 103 ]
( D)