Day 13: The Generative Zoo - Generative Adversarial Networks (GANs)

Day 13: The Generative Zoo - Generative Adversarial Networks (GANs)

Objective

Shift our focus from text to images and understand the architecture and training dynamics of Generative Adversarial Networks (GANs), a foundational model for image generation.

Core Concepts

• The GAN Framework:

  • GANs consist of two neural networks, a Generator and a Discriminator, that are trained simultaneously in a zero-sum game.
  • The Generator (The Counterfeiter):
    • Its job is to create fake, synthetic data (e.g., images) that looks like the real data.
    • It takes a random noise vector (called the latent vector, z) as input and outputs a fake image.
  • The Discriminator (The Detective):
    • Its job is to distinguish between real data (from the training set) and fake data (from the Generator).
    • It's a standard binary classifier that outputs a probability of the input image being real.

• The Adversarial Training Loop:

  1. The Generator creates a batch of fake images from random noise.
  2. The Discriminator is shown a mix of real images from the dataset and the fake images from the Generator.
  3. The Discriminator is trained to output 1 (Real) for the real images and 0 (Fake) for the fake images.
  4. The Generator is then trained. Its goal is to fool the Discriminator. It gets "rewarded" (its weights are updated) when the Discriminator incorrectly classifies its fake images as 1 (Real).
  5. This process repeats. Over time, the Generator gets better at creating realistic images, and the Discriminator gets better at telling them apart. The system reaches equilibrium when the Generator creates images that are so realistic the Discriminator can only guess with 50% accuracy.

🧠 Math & Stats Focus: The Minimax Game

The training process of a GAN is a minimax game between two players, the Generator (G) and the Discriminator (D).

• The Value Function V(D, G): The training objective can be expressed with a single value function. min_G max_D V(D, G) = E[log(D(x))] + E[log(1 - D(G(z)))]

  • E[...]: Expected value.
  • x: A real image from the data distribution.
  • z: A random noise vector from the latent space.
  • D(x): The Discriminator's probability that a real image x is real.
  • G(z): The Generator's output (a fake image).
  • D(G(z)): The Discriminator's probability that a fake image is real.

• The Game:

  • Maximizing for D: The Discriminator D wants to maximize this function. It does this by making D(x) close to 1 (correctly identifying real images) and D(G(z)) close to 0 (correctly identifying fake images). This makes both log(D(x)) and log(1 - D(G(z))) close to 0 (their maximum possible value).
  • Minimizing for G: The Generator G wants to minimize this function. It can only affect the second term. It tries to make D(G(z)) close to 1 (fooling the discriminator). This makes log(1 - D(G(z))) a large negative number, thus minimizing the overall function.

• Training Instability: This adversarial dynamic is powerful but notoriously unstable. Common failure modes include:

  • Mode Collapse: The Generator finds one or a few images that consistently fool the Discriminator and only produces those, instead of learning the full diversity of the dataset.
  • Vanishing Gradients: If the Discriminator becomes too good, it can provide no useful feedback (gradients) for the Generator to improve.

📜 Key Research Paper

• Paper: "Generative Adversarial Nets" (Goodfellow et al., 2014)
• Link: https://arxiv.org/abs/1406.2661
• Contribution: This paper introduced the entire GAN framework. The elegant idea of pitting two neural networks against each other in a zero-sum game was a conceptual breakthrough that kicked off a massive wave of research in generative modeling for images, leading to incredibly realistic results in the following years (e.g., StyleGAN, BigGAN).

💻 Project: Build a Simple GAN for MNIST

The "Hello, World!" of GANs is training one on the MNIST handwritten digits dataset.

1. Get the Data: Load the MNIST dataset from Keras/TensorFlow. Normalize the images to be in the range [-1, 1] (this often works better for GANs than [0, 1]).
2. Build the Generator:
   • It will take a latent vector (e.g., of size 100) as input.
   • It's essentially a "reverse" CNN. It uses Conv2DTranspose layers to upsample the input from the small latent vector to a 28x28 image.
   • Use LeakyReLU activations, and a tanh activation for the final output layer (to match the [-1, 1] normalization).
3. Build the Discriminator:
   • This is a standard binary classification CNN.
   • It takes a 28x28 image as input.
   • It uses Conv2D, LeakyReLU, Dropout, and Flatten layers.
   • The final output is a single neuron with a sigmoid activation.
4. Write the Custom Training Loop:
   • Unlike other models, GANs require a custom training loop because you need to alternate between training the discriminator and the generator.
   • You'll need to define two separate loss functions (e.g., BinaryCrossentropy) and two separate optimizers (e.g., Adam).
   • Following a detailed tutorial like the Keras DCGAN example is the best way to get this right.
5. Generate Images: Periodically save the output of your generator to see it improve from noisy static to recognizable digits over many epochs.

✅ Progress Tracker

• I can describe the roles of the Generator and the Discriminator.
• I understand the "adversarial" nature of the training process.
• I can explain what "mode collapse" is at a high level.
• I have attempted to build or have read through the code for a simple GAN.