Generative Adversarial Nets: Learning Data Distributions via Minimax Games

In 2014, Goodfellow et al. proposed the Generative Adversarial Network (GAN) framework. Its core idea is to estimate generative models through an adversarial process by simultaneously training two models: a generator G that captures the data distribution, and a discriminator D that estimates the probability that samples came from real data.

This framework corresponds to a minimax two-player game. In the space of arbitrary functions, there exists a unique solution where G recovers the training data distribution and D equals 1/2 everywhere. When G and D are defined by multilayer perceptrons, the entire system can be trained using backpropagation without requiring Markov chains or unrolled approximate inference networks.

The paper likens the generator to a team of counterfeiters trying to produce fake currency without being detected, while the discriminator acts like police trying to detect the counterfeit bills. This competition drives both parties to improve their methods until the fakes become indistinguishable from the genuine ones. This differs from traditional maximum likelihood estimation, which is often limited by difficulties in handling intractable probability computations.

Compared to Variational Autoencoders (VAEs), GANs require differentiability with respect to visible units and thus cannot model discrete data; whereas VAEs require differentiability with respect to hidden units and thus cannot have discrete latent variables. Additionally, the adversarial nature of GANs serves as the sole training criterion, sufficient to train the networks independently, unlike predictability minimization methods which act only as regularizers.

In practice, during early training, if G is poor, D will reject samples with high confidence, causing log(1-D(G(z))) to saturate. Therefore, the paper suggests training G to maximize log D(G(z)) instead of minimizing log(1-D(G(z))), providing stronger gradients in the early stages. This addresses numerical stability issues during optimization.