Advanced Deep Learning with Keras
[ 229 ]Chapter 7titles = ('MNIST predicted source images.','SVHN predicted target images.','MNIST reconstructed source images.','SVHN reconstructed target images.')dirs = ('mnist_source-%s' % postfix, 'svhn_target-%s' % postfix)# genrate predicted target(svhn) and source(mnist) imagesif g_models is not None:g_source, g_target = g_modelsother_utils.test_generator((g_source, g_target),(test_source_data, test_target_data),step=0,titles=titles,dirs=dirs,show=True)return# build the cyclegan for mnist cross svhnmodels = build_cyclegan(shapes,"mnist-%s" % postfix,"svhn-%s" % postfix,kernel_size=kernel_size,patchgan=patchgan)# patch size is divided by 2^n since we downscaled the input# in the discriminator by 2^n (ie. we use strides=2 n times)patch = int(source_data.shape[1] / 2**4) if patchgan else 1params = (batch_size, train_steps, patch, model_name)test_params = (titles, dirs)# train the cyclegantrain_cyclegan(models,data,params,test_params,other_utils.test_generator)The results for transferring the MNIST from the test dataset to SVHN are shownin Figure 7.1.12. The generated images have the style of SVHN, but the digits arenot completely transferred. For example, on the 4 th row, digits 3, 1, and 3 are stylizedby CycleGAN. However, on the 3 rd row, digits 9, 6, and 6 are stylized as 0, 6, 01, 0, 65,and 68 for the CycleGAN without and with PatchGAN respectively.The results of the backward cycle are shown in Figure 7.1.13. In this case, the targetimages are from the SVHN test dataset. The generated images have the style ofMNIST, but the digits are not correctly translated. For example, on the 1 st row, thedigits 5, 2, and 210 are stylized as 7, 7, 8, 3, 3, and 1 for the CycleGAN without andwith PatchGAN respectively.
Cross-Domain GANsIn the case of PatchGAN, the output 1 is understandable given the predicted MNISTdigit is constrained to one digit. There are somehow correct predictions like in 2 ndrow last 3 columns of the SVHN digits, 6, 3, and 4 are converted to 6, 3, and 6 byCycleGAN without PatchGAN. However, the outputs on both flavors of CycleGANare consistently single digit and recognizable.The problem exhibited in the conversion from MNIST to SVHN where a digit inthe source domain is translated to another digit in the target domain is called labelflipping [8]. Although the predictions of CycleGAN are cycle-consistent, they arenot necessarily semantic consistent. The meaning of digits is lost during translation.To address this problem, Hoffman [8] introduced an improved CycleGAN calledCyCADA (Cycle-Consistent Adversarial Domain Adaptation). The difference is theadditional semantic loss term ensures that the prediction is not only cycle-consistentbut also sematic-consistent:Figure 7.1.12: Style transfer of test data from the MNIST domain to SVHN. Original color photo can be foundon the book GitHub repository, https://github.com/PacktPublishing/Advanced-Deep-Learning-with-Keras/blob/master/chapter7-cross-domain-gan/README.md.[ 230 ]
- Page 195 and 196: Disentangled Representation GANspyt
- Page 197 and 198: Disentangled Representation GANsThe
- Page 199 and 200: Disentangled Representation GANsSta
- Page 201 and 202: Disentangled Representation GANs( )
- Page 203 and 204: Disentangled Representation GANsThe
- Page 205 and 206: Disentangled Representation GANsfea
- Page 207 and 208: Disentangled Representation GANs# f
- Page 209 and 210: Disentangled Representation GANslat
- Page 211 and 212: Disentangled Representation GANsDis
- Page 213 and 214: Disentangled Representation GANsz_d
- Page 215 and 216: Disentangled Representation GANs2.
- Page 217 and 218: Disentangled Representation GANsFig
- Page 220 and 221: Cross-Domain GANsIn computer vision
- Page 222 and 223: Chapter 7There are many more exampl
- Page 224 and 225: The CycleGAN ModelFigure 7.1.3 show
- Page 226 and 227: Chapter 7Repeat for n training step
- Page 228 and 229: Chapter 7Implementing CycleGAN usin
- Page 230 and 231: filters=16,kernel_size=3,strides=2,
- Page 232 and 233: Chapter 7kernel_size=kernel_size)e3
- Page 234 and 235: Listing 7.1.3, cyclegan-7.1.1.py sh
- Page 236 and 237: Chapter 71) Build target and source
- Page 238 and 239: Chapter 7preal_target,reco_source,r
- Page 240 and 241: size=batch_size)real_source = sourc
- Page 242 and 243: Chapter 7returndirs=dirs,show=True)
- Page 244 and 245: Chapter 7Figure 7.1.10: Color (from
- Page 248 and 249: Chapter 7Figure 7.1.13: Style trans
- Page 250 and 251: Chapter 7Figure 7.1.15: The backwar
- Page 252: Chapter 7References1. Yuval Netzer
- Page 255 and 256: Variational Autoencoders (VAEs)In t
- Page 257 and 258: Variational Autoencoders (VAEs)Typi
- Page 259 and 260: Variational Autoencoders (VAEs)For
- Page 261 and 262: Variational Autoencoders (VAEs)VAEs
- Page 263 and 264: Variational Autoencoders (VAEs)outp
- Page 265 and 266: Variational Autoencoders (VAEs)Figu
- Page 267 and 268: Variational Autoencoders (VAEs)The
- Page 269 and 270: Variational Autoencoders (VAEs)Figu
- Page 271 and 272: Variational Autoencoders (VAEs)Prec
- Page 273 and 274: Variational Autoencoders (VAEs)shap
- Page 275 and 276: Variational Autoencoders (VAEs)cvae
- Page 277 and 278: Variational Autoencoders (VAEs)Figu
- Page 279 and 280: Variational Autoencoders (VAEs)Figu
- Page 281 and 282: Variational Autoencoders (VAEs)In F
- Page 283 and 284: Variational Autoencoders (VAEs)Figu
- Page 285 and 286: Variational Autoencoders (VAEs)The
- Page 288 and 289: Deep ReinforcementLearningReinforce
- Page 290 and 291: [ 273 ]Chapter 9Formally, the RL pr
- Page 292 and 293: Chapter 9Where:( ) ( , )∗V s maxQ
- Page 294 and 295: Chapter 9Initially, the agent assum
[ 229 ]
Chapter 7
titles = ('MNIST predicted source images.',
'SVHN predicted target images.',
'MNIST reconstructed source images.',
'SVHN reconstructed target images.')
dirs = ('mnist_source-%s' % postfix, 'svhn_target-%s' % postfix)
# genrate predicted target(svhn) and source(mnist) images
if g_models is not None:
g_source, g_target = g_models
other_utils.test_generator((g_source, g_target),
(test_source_data, test_
target_data),
step=0,
titles=titles,
dirs=dirs,
show=True)
return
# build the cyclegan for mnist cross svhn
models = build_cyclegan(shapes,
"mnist-%s" % postfix,
"svhn-%s" % postfix,
kernel_size=kernel_size,
patchgan=patchgan)
# patch size is divided by 2^n since we downscaled the input
# in the discriminator by 2^n (ie. we use strides=2 n times)
patch = int(source_data.shape[1] / 2**4) if patchgan else 1
params = (batch_size, train_steps, patch, model_name)
test_params = (titles, dirs)
# train the cyclegan
train_cyclegan(models,
data,
params,
test_params,
other_utils.test_generator)
The results for transferring the MNIST from the test dataset to SVHN are shown
in Figure 7.1.12. The generated images have the style of SVHN, but the digits are
not completely transferred. For example, on the 4 th row, digits 3, 1, and 3 are stylized
by CycleGAN. However, on the 3 rd row, digits 9, 6, and 6 are stylized as 0, 6, 01, 0, 65,
and 68 for the CycleGAN without and with PatchGAN respectively.
The results of the backward cycle are shown in Figure 7.1.13. In this case, the target
images are from the SVHN test dataset. The generated images have the style of
MNIST, but the digits are not correctly translated. For example, on the 1 st row, the
digits 5, 2, and 210 are stylized as 7, 7, 8, 3, 3, and 1 for the CycleGAN without and
with PatchGAN respectively.