DCGAN - Deep Convolution GAN


INTRODUCTION:

In recent research, we have seen multiple CNN classification, detection and segmentation algorithms such as Inception versions, YOLO versions, FCN, U-net and KSAC.

All high accuracy model requires a large amount of data along with labelling. In the real world, data are usually in the unlabeled form. So, data gathering and labelling are really time-consuming and erroneous.

DCGAN can be helpful to learn intermediate features from the unlabeled data. The feature representation can be leveraged for further all types of supervised tasks.

One might know that GAN contains two types of architecture:

  1. Discriminator
  2. Generator
After training of GAN:
  • Discriminators can be used for image classification tasks.
  • Generators can be used to manipulate the semantic qualities of generated images.

DETAIL OF DCGAN:

DCGAN is basically used to stable GANs for convolution in most settings. It also helps to visualize filters that have been learnt to detect specific shapes. Using generator input vector changes, One can manipulate generated object qualities.

Approach:

There are a total of 4 modifications done to make the model more stable and accurate.
  1. use LAPGAN:
    • It helps to upscale low resolution generated images.
  2. Replace max pooling with strides:
    • Allow the network to learn its own spatial downsampling.
  3. eliminate fully connected layers with the global average pooling layer:
    • Helps to increase stability but reduce the convergence speed.
  4. Use batch-normalization:
    • Gets the benefit of stabilized learning
    • Helps in poor weights initialization and gradient flow in deeper models.

Architecture:

Block Diagram:

Figure 1: DCGAN block diagram

Discriminator:

Figure 2: Discriminator architecture

Generator:

The generator uses Laplacian GAN architecture to generate high-quality samples of natural images.

Figure 3: Generator architecture

Training:

The model is trained using the following hyperparameters:
  • mini-batch stochastic gradient descent.
  • batch size = 128
  • weight initialization: zero centred normal distribution with 0.02 standard deviation.
  • slope of leaky relu = 0.2
  • Adam optimzer
  • The generator uses Laplacian GAN architecture to generate high-quality samples of natural images. The generator uses Laplacian GAN architecture to generate high-quality samples of natural images.learning rate = 0.0002
  • B1 = 0.5

Adversarial Loss:

In Every GAN model, Adversarial loss is a must. Adversarial loss tries to match the distribution of generated images to dataset distribution.

Where D tries to maximize the correctness with the help of logD(x) and G tries to minimize the fake prediction with the help of log(1 -D(G(z))).

So, Adversarial loss is used in any GAN network. The loss function applies to the output of that discriminator.

Label Smoothing:

Label smoothing is a regularised technique that introduces noise of the labels which helps stabilize DCGAN training.

This accounts for the fact that datasets may have mistakes in them, so maximizing the likelihood of log P(y|x) directly can be harmful.

Assume for a small constant ϵ, the training set label y is correct with probability 1−ϵ and incorrect otherwise. Label Smoothing regularizes a model based on a binary classification by replacing the hard 0 and 1 classification targets with targets of ϵ−1 and 1−ϵ respectively.

Figure 4: label smoothing for 3 classes


CONCLUSION:

The Blog explained about basic understanding of DCGAN and architectures of discriminator and generator. The main focus is the stable network that can train any type of image generation with such specified classes. There are also architectural changes and training processing functionalities such as label smoothing and adversarial loss functions. It also mentioned basic training hyperparameters to start with.

Comments

Post a Comment

Popular posts from this blog

CyclicGAN

Image
  INTRODUCTION Image to image conversion is a really hard task to do when you don't have any specific output label image. There are also other approaches as follows: Unpair image to image translation  can translate one image to another but the model can be trained on a specific translation. Neural style transfer translates input images on a specific painting or style. but it can not convert from a style to back to the original. Using cyclic GAN , an image can convert to any other style passed in the model at that time as a reference style image. and the model also helps to convert back from a style to original by just changing the location of passing images to the model. WORK OF CYCLIC-GAN The main goal of the cyclic model is to map image A to image B and vice versa. The cyclic model contains two GAN models one map image A to image B and another map B image to image A. G : X ↦ Y ⇒ G ( X ) F : Y ↦ X ⇒ F ( G ( X ) ) ≈ X Where,      G is a GAN model that maps image...
Read more

Difference between DCGAN and cyclicGAN

Image
Introduction: In recent times, lots of research have been going on, and many new techniques have been introduced in the Deep Learning domain. Especially, the development has been made in the computer vision area such as Deepface and  3D model generation  using 2d images and many more. There is also a subdomain of vision area which can generate unseen images by training a Generative adversarial network(GAN). There are multiple ways to generate images depending on applications. But, there are two models which can be used mostly. DCGAN (Deep convolution GAN) cyclicGAN Explanation: One can check the following criteria to distinguish between these two models. Architecture: DCGAN has basic GAN architecture which contains a generator and discriminator. The input of the generator is a random numbers array from any distribution (normal/uniform). Figure 1:   DCGAN functional diagram CyclicGAN has a complex architecture that contains two generators and discriminators. One GAN archi...
Read more