Advanced Deep Learning with Keras
Share Embed Flag
Download ePaper

Advanced Deep Learning with Keras

fourpersent2020
from fourpersent2020 More from this publisher
16.03.2021 Views

Chapter 6For continuous codes of a single dimension, the expectation is a double integralover c and x. This is due to the expectation that samples from both disentangledcode distribution and generator distribution. One way of estimating the expectationis by assuming the samples as a good measure of continuous data. Therefore, the lossis estimated as c log Q(c|x).To complete the network of InfoGAN, we should have an implementation of Q(c|x).For simplicity, the network Q is an auxiliary network attached to the second to lastlayer of the discriminator. Therefore, this has a minimal impact on the training of theoriginal GAN. Following figure shows InfoGAN network diagram:Figure 6.1.2: A network diagram showing the discriminator and generator training in InfoGAN[ 165 ]

Disentangled Representation GANsFollowing table shows the loss functions of InfoGAN as compared to the originalGAN. The loss functions of InfoGAN differ from the original GAN by an additionalterm −λI ( c; G ( z,c)) where λ is a small positive constant. Minimizing the lossfunction of InfoGAN translates to minimizing the loss of the original GAN andI c; G z,c .maximizing the mutual information ( ( ))Network Loss Functions NumberGAN ( D)L = −Elog D x − E log 1− D G z4.1.1L( G)x∼pdataz( ( ))= −Elog D G zz( )( ) ( ( ))( ( )) ( ( ))InfoGAN ( D)L = −Ex∼plog D ( x) − Ez,clog 1 − D G ( z, c)− λI c; G z,c( G)data( ( )) ( ( ))L = −Ez, clog D G z, c − λI c; G z,cλ < . In ourFor continuous codes, InfoGAN recommends a value of 1example, we set λ = 0.5 . For discrete codes, InfoGAN recommendsλ = 1.Table 6.1.1: A comparison between the loss functions of GAN and InfoGANIf applied on the MNIST dataset, InfoGAN can learn the disentangled discrete andcontinuous codes in order to modify the generator output attributes. For example,like CGAN and ACGAN, the discrete code in the form of a 10-dim one-hot label willbe used to specify the digit to generate. However, we can add two continuous codes,one for controlling the angle of writing style and another for adjusting the strokewidth. Following figure shows the codes for the MNIST digit in InfoGAN. We retainthe entangled code with a smaller dimensionality to represent all other attributes:4.1.56.1.16.1.2Figure 6.1.3: The codes for both GAN and InfoGAN in the context of MNIST dataset[ 166 ]

Chapter 6

For continuous codes of a single dimension, the expectation is a double integral

over c and x. This is due to the expectation that samples from both disentangled

code distribution and generator distribution. One way of estimating the expectation

is by assuming the samples as a good measure of continuous data. Therefore, the loss

is estimated as c log Q(c|x).

To complete the network of InfoGAN, we should have an implementation of Q(c|x).

For simplicity, the network Q is an auxiliary network attached to the second to last

layer of the discriminator. Therefore, this has a minimal impact on the training of the

original GAN. Following figure shows InfoGAN network diagram:

Figure 6.1.2: A network diagram showing the discriminator and generator training in InfoGAN

[ 165 ]

logo

products

Resources

Company

Terms of service
Privacy policy
Cookie policy
Cookie settings
Imprint
Change language
Made with love in Switzerland
© 2024 Yumpu.com all rights reserved

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!