CyclicGAN

 

INTRODUCTION

Image to image conversion is a really hard task to do when you don't have any specific output label image.

There are also other approaches as follows:

Unpair image to image translation can translate one image to another but the model can be trained on a specific translation.

Neural style transfer translates input images on a specific painting or style. but it can not convert from a style to back to the original.

Using cyclic GAN, an image can convert to any other style passed in the model at that time as a reference style image. and the model also helps to convert back from a style to original by just changing the location of passing images to the model.

WORK OF CYCLIC-GAN

The main goal of the cyclic model is to map image A to image B and vice versa.

The cyclic model contains two GAN models one map image A to image B and another map B image to image A.

$$G:X\mapsto Y\Rightarrow G(X)$$

$$F:Y\mapsto X\Rightarrow F(G(X))\approx X$$

Where,

    G is a GAN model that maps image X to image Y.

    F is a GAN model that maps image Y to image X.

Block Diagram

Figure 1 shows that a cycle of image A passed to Gx and Gy generators to try to get a similar image to A with the training of both generators and one discriminator. Generator X helps to map image A to image B and discriminator helps to correct generated image from Gx only.

Figure 1: A cycle block digram for image A 

Figure 2 shows that a cycle of image B passed to both generators to try to get a similar image to B with the training of both generators and one discriminator. Generator X helps to map image B to image A and discriminator helps to correct generated image from Gy only.

Figure 2: A cycle block diagram for image B

Cyclic consistency loss can help to understand the approach to create cyclic output for both images. For more detail, one can check the cyclic consistency loss in the loss function section.

Architecture

Discriminator:

Figure 3: Discriminator Model

Generator:

Figure 4: Generator Model

Where,

    k = kernel size

    s = strides

    p = padding

    Li = layer number

Loss functions

In the usage of loss functions, The adversarial loss function is required in the CyclicGAN model to train GAN models to generate realistic images. The cyclic consistency loss function is used in  CyclicGAN that can help to transfer the style from Image A to B and vice versa.

Adversarial loss:

In Every GAN model, Adversarial loss tries to match the distribution of generated images to dataset distribution.

V(D,G)=Ex∼pdata​(x)​[logD(x)]+Ez∼pz​(z)​[log(1 -D(G(z)))]

Where D tries to maximize the correctness with the help of logD(x) and G tries to minimize the fake prediction with the help of log(1 -D(G(z))).

So, Adversarial loss is used in both Cycles. The loss function applies to the output of Dx and Dy discriminators.

Cyclic consistency loss:

Cyclic consistency loss is basically used to push both mapping models cycle model for image A and cyclic model for image B to be consistent with each other. It helps to prevent mapping by contradicting each other.

Forward cyclic consistency (for image A):

$$x\to G(x)\to F(G(x))\approx x$$

Cyclic consistency loss function:

$$L_{cyclic}(G,F)=E_{x\sim p_{data}(x)}[\Vert F(G(x))-x{\Vert }_1]+E_{y\sim p_{data}(y)}[\Vert G(F(y))-y{\Vert }_1]$$

The CyclicGAN has two auto-encoders that are joint to each other.

$$F\circ G:X\mapsto X$$

Both autoencoders have an internal structure such that it maps image A to itself using one cycle shown in figure 1. Image B cyclic output will be the translation of image A into image B domain.

CONCLUSION

This article explains every part of cyclicGAN in detail. specifically, break downed functionalities for both GAN models which map one image to another. It also displayed the NN architecture structure for discriminator and generator. explain adversarial loss will help to check the model generates actual or fake images. Cyclic consistency loss will help to prevent mapping by contradicting each other.