Project · Generative · 2024 · solo
Discriminator rejection sampling for GANs
Four variants of discriminator rejection sampling on MNIST, from a naive rule to adaptive truncation, and the quality–diversity trade-off each makes.
Problem
GANs still produce artefacts and miss modes. Discriminator Rejection Sampling (Azadi et al., 2019) keeps each generated sample with a probability that depends on the discriminator’s score D(x). The question is how to set that probability so quality improves without diversity collapsing.
Method
A fully connected GAN on MNIST: a four-layer generator and a three-layer discriminator (a weaker discriminator turned out to be a better filter). I built four variants in turn: basic DRS with a global normaliser, a ratio-based rule recomputed per batch, DRS with soft truncation of the latent code, and an adaptive soft truncation that only touches samples the discriminator scores poorly.
Results
Soft truncation buys the best precision at the cost of the worst recall; the adaptive variant keeps most of that precision and reaches the highest recall of the four. A Metropolis–Hastings sampler is the natural next step.
Measurements
| Setting | Value | Note |
|---|---|---|
| Basic DRS (global M) | P 0.48–0.50 · R 0.21 | |
| Ratio-based DRS (per-batch M) | P 0.51–0.52 · R 0.22 | |
| DRS + soft truncation | P 0.55 · R 0.16 | |
| DRS + adaptive soft truncation | P 0.53 · R 0.24 | best balance |
Precision and recall of the generated distribution for each sampling variant (project report).
Stack
Python · PyTorch · torchvision · NumPy · scikit-learn