[Deep learning] Recent development of Generative Adversarial Networks (GANs)

Introduction of cGANs, pix2pix, and Cycle GANs

In this post, I would like to discuss the recent development in GANs.

Conditional GANs (cGANs)

The original GANs trains the generator and discriminator with no supplementary information. Therefore, Conditional GANs(cGANs)[5] adds some condition constraints to GANs such that the model is more capable of handling different contextual information.

The structure of cGANs from the original paper from the original paper

From the above figure in the original paper[5], both the generator and discriminator in cGANs include the additional condition y. For example, the paper shows that cGANs trained from MNIST images (database of handwritten digits) with its digit labels. The updated objective function is as follow:

Image by the author

After the foundation of cGAN, there was a popular paper, ”Image style transfer using convolutional neural networks”[3], which introduced the application of style transfer by GANs. Later, there were several GANs methods for domain transfer, such as ix2pix GANs, Cycle GANs, Perceptual Adversarial Networks and Star GANs etc. In the next two subsections, Cycle GANs and pix2pix will be discussed.

pix2pix

Pix2pix[7] is a cGANs based model, which aims to perform ”Image-to-Image” translation. One of the drawbacks mentioned in the other generated models is that they tend to generate blurry images because the image is generated pixel by pixel. Instead, pix2pix considers the whole structure of the image.

The methodology of pix2pix from the original paper

Pix2pix requires paired training data. For example, in the above figure, both the generator and discriminator pix2pix requires both the real shoes (y ) and the framework of that shoe (as x). That is to say, the generator does not start from scratch, but starting with the help of the picture framework (an additional condition in cGANs). Oh the other hand, the…