The Gyan of GAN
I presented a Paper during the proceedings of ICCIDE2023 on Log Parsing. During the same, I heard Dr. Ravi Vadlamani’s lecture on FinGAN, an adaptation/extension of ChaoticGAN. My interest grew around this excellent technical possibility. When the need arose to submit a project as part of the fulfillment of the WE course by TalentSprint, I chose to implement basic GAN to feel the beauty of the same.
The general Neural Networks can easily misclassify things by adding even a tiny amount of noise into the original data. When noise is introduced, the model’s confidence in an incorrect prediction is unexpectedly higher. This phenomenon occurs because many machine learning models are trained on a limited amount of data, which makes them susceptible to overfitting. Additionally, the relationship between input and output is often almost linear. While the separation boundaries between classes may appear linear, they are made up of linearities, and even a small alteration to a feature point can result in misclassification.
Generative Adversarial Networks (GANs) address the above challenge. GANs are a type of deep learning architecture comprising two neural networks: a generator and a discriminator. The generator network creates new data samples, such as images or audio, while the discriminator network distinguishes between the generated and actual samples from a dataset. The beauty of the technique is that networks are trained together in an adversarial manner, where the generator tries to fool the discriminator, and the discriminator tries to classify the samples accurately.
This simple implementation of fundamental human psychology: learning through imitation — and the reflection of Darwin’s doctrine of “Survival of the fittest” observed during the training is what amazed me the most — the generator is trained to survive — and it takes me back to appreciating and acknowledging the beauty in simplicity.
The basic idea of GANs is to generate realistic samples in terms of a probabilistic model that are indistinguishable from real examples without explicitly modeling the probability distribution of the data. Instead, GANs learn to approximate the distribution by training the generator network to generate samples similar to the actual data using Deep Neural Networks as AI algorithms for training.
In this context, the generative model is designed to capture the data distribution and is trained to maximize the probability of the Discriminator making an error. In contrast, the Discriminator model estimates the likelihood of a given sample originating from the training data rather than the Generator. GANs are constructed as a minimax game in which the Discriminator seeks to minimize its reward V(D, G). At the same time, the Generator aims to reduce the Discriminator’s reward, which is equivalent to maximizing its loss. This can be expressed mathematically using the following formula:
D(x;θd) is Discriminator Model, G(x;θg) is the generator model, pdata data is the distribution of the data, pz is the distribution of the noise
Several types of GANs have been developed for different data types and applications. Some of the most common types include:
- Deep Convolutional GANs (DCGANs): These GANs use convolutional neural networks (CNNs) for image generation.
- Conditional GANs (CGANs): These are GANs that take in additional information, such as labels or class information, to generate samples that belong to a specific class or category.
- CycleGANs: These are GANs that learn to translate images from one domain to another without using paired data for training.
- Wasserstein GANs (WGANs): These GANs use a different loss function, the Wasserstein distance, to improve stability and avoid mode collapse.
GANs have many applications in computer vision, natural language processing, and audio processing. Some examples of where GANs can be applied include:
- Image generation and editing: GANs can generate new images similar to a given dataset or edit existing images by changing specific attributes, such as color or style.
- Video generation and prediction: GANs can generate realistic video frames or predict future frames based on a sequence of inputs.
- Data augmentation or Synthetic Data Generation: GANs can be used to generate new data samples that can be used to augment existing datasets, and anomaly or aberration detection, which can improve the performance of machine learning models. Aberration detection has implications in finding ‘security/intrusion issues, etc.
- Text-to-image synthesis: GANs can generate images from text descriptions, such as creating a visual representation of a scene described in a sentence.
The advantages of GAN include:
- Unsupervised learning: GANs can be trained without labeled data, making them suitable for unsupervised learning tasks where labeled data is scarce or difficult to obtain.
- Transfer learning: GANs can also be used for transfer learning, using a pre-trained generator to generate data in a new domain. For example, a GAN trained on images of dogs could be used to generate images of cats without starting from scratch.
- Data augmentation: GANs can generate additional synthetic data to supplement the limited actual data. This can be particularly useful when training machine learning models, as it can improve the accuracy and generalization of the models.
- Improved privacy: GANs can be used to generate synthetic data that is similar to actual data but does not contain any personally identifiable information. This can be useful for protecting privacy when working with sensitive data.
This blog, in particular, reflects what I’ve learned so far. This concept is something I’m still learning about and meditating on. As I go forward, I’ll keep posting my insights on the same.
As I keep learning the Gyan of GAN — I’ll try to impart it here. Until then — Adios!!
