Advanced Deep Learning with Keras

Improved GANs
This makes sense since the objective of the generator is to fool the discriminator by
learning the true data distribution. Effectively, we can arrive at the optimal generator
by minimizing D JS
, or by making p g
→ p data
. Given an optimal generator, the optimal
( D*
)
discriminator is with L = 2log 2 = 0.60.
Figure 5.1.2: An example of two distributions with no overlap. θ = 0.5 for p g
The problem is that when the two distributions have no overlap, there's no smooth
function that will help to close the gap between them. Training the GANs will not
converge by gradient descent. For example, let's suppose:
p data
=(x, y) where x 0, y ~ U ( 0,1)
= (Equation 5.1.14)
p g
= (x, y) where x θ, y ~ U ( 0,1)
= (Equation 5.1.15)
As shown in Figure 5.1.2. U(0,1) is the uniform distribution. The divergence for each
distance function is as follows:
• DKL ( pg pdata) Ex=θ, y~ U( 0,1)
• DKL ( pg pdata) x y U( )
pg
( x, y)
1
= log = ∑1log
=+∞
pdata
( x, y)
0
pg
( x, y)
1
= E log = 1log
=θ, ~ 0,1 ∑ =+∞
p ( x, y)
0
data
[ 130 ]