Advanced Deep Learning with Keras
Chapter 7Implementing CycleGAN using KerasLet us tackle a simple problem that CycleGAN can address. In Chapter 3,Autoencoders, we used an autoencoder to colorize grayscale images from theCIFAR10 dataset. We can recall that the CIFAR10 dataset is made of 50,000 traineddata and 10,000 test data samples of 32 × 32 RGB images belonging to ten categories.We can convert all color images into grayscale using rgb2gray(RGB) as discussed inChapter 3, Autoencoders.Following on from that, we can use the grayscale train images as source domainimages and the original color images as the target domain images. It's worth notingthat although the dataset is aligned, the input to our CycleGAN is a random sampleof color images and a random sample of grayscale images. Thus, our CycleGAN willnot see the train data as aligned. After training, we'll use the test grayscale imagesto observe the performance of the CycleGAN:Figure 7.1.6: The forward cycle generator G, implementation in Keras.The generator is a U-Network made of encoder and decoder.[ 211 ]
Cross-Domain GANsAs discussed in the previous section, to implement the CycleGAN, we need to buildtwo generators and two discriminators. The generator of CycleGAN learns the latentrepresentation of the source input distribution and translates this representation intotarget output distribution. This is exactly what autoencoders do. However, typicalautoencoders similar to the ones discussed in Chapter 3, Autoencoders, use an encoderthat downsamples the input until the bottleneck layer at which point the processis reversed in the decoder. This structure is not suitable in some image translationproblems since many low-level features are shared between the encoder and decoderlayers. For example, in colorization problems, the form, structure, and edges of thegrayscale image are the same as in the color image. To circumvent this problem,the CycleGAN generators use a U-Net [7] structure as shown in Figure 7.1.6.In a U-Net structure, the output of the encoder layer e n-iis concatenated withthe output of the decoder layer d i, where n = 4 is the number of encoder/decoderlayers and i = 1, 2 and 3 are layer numbers that share information.We should note that although the example uses n = 4, problems with a higher input/output dimensions may require deeper encoder/decoder. The U-Net structureenables a free flow of feature-level information between encoder and decoder.An encoder layer is made of Instance Normalization(IN)-LeakyReLU-Conv2Dwhile the decoder layer is made of IN-ReLU-Conv2D. The encoder/decoder layerimplementation is shown in Listing 7.1.1 while the generator implementation isshown in Listing 7.1.2.The complete code is available on GitHub:https://github.com/PacktPublishing/Advanced-Deep-Learning-with-KerasInstance Normalization (IN) is Batch Normalization (BN) per sample of data(that is, IN is BN per image or per feature). In style transfer, it's important tonormalize the contrast per sample not per batch. Instance normalization isequivalent to contrast normalization. Meanwhile, Batch normalization breakscontrast normalization.Remember to install keras-contrib before using instance normalization:$ sudo pip3 install git+https://www.github.com/keras-team/keras-contrib.gitListing 7.1.1, cyclegan-7.1.1.py shows us the encoder and decoder layersimplementation in Keras:def encoder_layer(inputs,[ 212 ]
- Page 177 and 178: Improved GANsConclusionIn this chap
- Page 179 and 180: Disentangled Representation GANsIn
- Page 181 and 182: Disentangled Representation GANsInf
- Page 183 and 184: Disentangled Representation GANsFol
- Page 185 and 186: Disentangled Representation GANs# A
- Page 187 and 188: Disentangled Representation GANsif
- Page 189 and 190: Disentangled Representation GANsLis
- Page 191 and 192: Disentangled Representation GANsdat
- Page 193 and 194: Disentangled Representation GANsy[b
- Page 195 and 196: Disentangled Representation GANspyt
- Page 197 and 198: Disentangled Representation GANsThe
- Page 199 and 200: Disentangled Representation GANsSta
- Page 201 and 202: Disentangled Representation GANs( )
- Page 203 and 204: Disentangled Representation GANsThe
- Page 205 and 206: Disentangled Representation GANsfea
- Page 207 and 208: Disentangled Representation GANs# f
- Page 209 and 210: Disentangled Representation GANslat
- Page 211 and 212: Disentangled Representation GANsDis
- Page 213 and 214: Disentangled Representation GANsz_d
- Page 215 and 216: Disentangled Representation GANs2.
- Page 217 and 218: Disentangled Representation GANsFig
- Page 220 and 221: Cross-Domain GANsIn computer vision
- Page 222 and 223: Chapter 7There are many more exampl
- Page 224 and 225: The CycleGAN ModelFigure 7.1.3 show
- Page 226 and 227: Chapter 7Repeat for n training step
- Page 230 and 231: filters=16,kernel_size=3,strides=2,
- Page 232 and 233: Chapter 7kernel_size=kernel_size)e3
- Page 234 and 235: Listing 7.1.3, cyclegan-7.1.1.py sh
- Page 236 and 237: Chapter 71) Build target and source
- Page 238 and 239: Chapter 7preal_target,reco_source,r
- Page 240 and 241: size=batch_size)real_source = sourc
- Page 242 and 243: Chapter 7returndirs=dirs,show=True)
- Page 244 and 245: Chapter 7Figure 7.1.10: Color (from
- Page 246 and 247: [ 229 ]Chapter 7titles = ('MNIST pr
- Page 248 and 249: Chapter 7Figure 7.1.13: Style trans
- Page 250 and 251: Chapter 7Figure 7.1.15: The backwar
- Page 252: Chapter 7References1. Yuval Netzer
- Page 255 and 256: Variational Autoencoders (VAEs)In t
- Page 257 and 258: Variational Autoencoders (VAEs)Typi
- Page 259 and 260: Variational Autoencoders (VAEs)For
- Page 261 and 262: Variational Autoencoders (VAEs)VAEs
- Page 263 and 264: Variational Autoencoders (VAEs)outp
- Page 265 and 266: Variational Autoencoders (VAEs)Figu
- Page 267 and 268: Variational Autoencoders (VAEs)The
- Page 269 and 270: Variational Autoencoders (VAEs)Figu
- Page 271 and 272: Variational Autoencoders (VAEs)Prec
- Page 273 and 274: Variational Autoencoders (VAEs)shap
- Page 275 and 276: Variational Autoencoders (VAEs)cvae
- Page 277 and 278: Variational Autoencoders (VAEs)Figu
Chapter 7
Implementing CycleGAN using Keras
Let us tackle a simple problem that CycleGAN can address. In Chapter 3,
Autoencoders, we used an autoencoder to colorize grayscale images from the
CIFAR10 dataset. We can recall that the CIFAR10 dataset is made of 50,000 trained
data and 10,000 test data samples of 32 × 32 RGB images belonging to ten categories.
We can convert all color images into grayscale using rgb2gray(RGB) as discussed in
Chapter 3, Autoencoders.
Following on from that, we can use the grayscale train images as source domain
images and the original color images as the target domain images. It's worth noting
that although the dataset is aligned, the input to our CycleGAN is a random sample
of color images and a random sample of grayscale images. Thus, our CycleGAN will
not see the train data as aligned. After training, we'll use the test grayscale images
to observe the performance of the CycleGAN:
Figure 7.1.6: The forward cycle generator G, implementation in Keras.
The generator is a U-Network made of encoder and decoder.
[ 211 ]