Advanced Deep Learning with Keras

Cross-Domain GANs
In computer vision, computer graphics, and image processing a number of tasks
involve translating an image from one form to another. As an example, colorization
of grayscale images, converting satellite images to maps, changing the artwork style
of one artist to another, making night-time images into daytime, and summer photos
to winter, are just a few examples. These tasks are referred to as cross-domain
transfer and will be the focus of this chapter. An image in the source domain
is transferred to a target domain resulting in a new translated image.
A cross-domain transfer has a number of practical applications in the real world.
As an example, in autonomous driving research, collecting road scene driving data
is both time-consuming and expensive. In order to cover as many scene variations
as possible in that example, the roads would be traversed during different weather
conditions, seasons, and times giving us a large and varied amount of data. With the
use of a cross-domain transfer, it's possible to generate new synthetic scenes that look
real by translating existing images. For example, we may just need to collect road
scenes in the summer from one area and gather road scenes in the winter from
another place. Then, we can transform the summer images to winter and the winter
images to summer. In this case, it reduces the number of tasks having to be done
by half.
Generation of realistic synthesized images is an area that GANs excel at. Therefore,
cross-domain translation is one of the applications of GANs. In this chapter, we're
going to focus on a popular cross-domain GAN algorithm called CycleGAN [2].
Unlike other cross-domain transfer algorithms, such as a pix2pix [3], CycleGAN
doesn't require aligned training images to work. In aligned images, the training data
should be a pair of images made up of the source image and its corresponding target
image. For example, a satellite image and the corresponding map derived from this
image. CycleGAN only requires the satellite data images and maps. The maps may
be from another satellite data and are not necessarily previously generated from the
training data.
[ 203 ]