Advanced Deep Learning with Keras

Disentangled Representation GANs
Following table shows the loss functions of InfoGAN as compared to the original
GAN. The loss functions of InfoGAN differ from the original GAN by an additional
term −λI ( c; G ( z,
c)
) where λ is a small positive constant. Minimizing the loss
function of InfoGAN translates to minimizing the loss of the original GAN and
I c; G z,
c .
maximizing the mutual information ( ( ))
Network Loss Functions Number
GAN ( D)
L = −E
log D x − E log 1− D G z
4.1.1
L
( G)
x∼
pdata
z
( ( ))
= −E
log D G z
z
( )
( ) ( ( ))
( ( )) ( ( ))
InfoGAN ( D)
L = −Ex∼
p
log D ( x) − Ez,
c
log 1 − D G ( z, c)
− λI c; G z,
c
( G)
data
( ( )) ( ( ))
L = −Ez, c
log D G z, c − λI c; G z,
c
λ < . In our
For continuous codes, InfoGAN recommends a value of 1
example, we set λ = 0.5 . For discrete codes, InfoGAN recommends
λ = 1.
Table 6.1.1: A comparison between the loss functions of GAN and InfoGAN
If applied on the MNIST dataset, InfoGAN can learn the disentangled discrete and
continuous codes in order to modify the generator output attributes. For example,
like CGAN and ACGAN, the discrete code in the form of a 10-dim one-hot label will
be used to specify the digit to generate. However, we can add two continuous codes,
one for controlling the angle of writing style and another for adjusting the stroke
width. Following figure shows the codes for the MNIST digit in InfoGAN. We retain
the entangled code with a smaller dimensionality to represent all other attributes:
4.1.5
6.1.1
6.1.2
Figure 6.1.3: The codes for both GAN and InfoGAN in the context of MNIST dataset
[ 166 ]