Classification with Multiple Generative Adversaries

In this work, we consider a natural extension of the two-player adversarial framework proposed by Goodfellow et al. (2014), specifically, in our setting a single discriminative network will compete against multiple generative networks, with each attempting to produce realistic samples from one of N distinct distributions. The objective of the Discriminator will be to classify every given input to one of 2N categories: N real classes and their fake counterparts. As a result, our adversarial paradigm is explicitly formulated in order to train the Discriminator to perform classification. We provide a theoretical analysis of our proposed architecture, deriving an expression for the optimal Discriminator and investigating the best response from the Generators - under optimal play. Moreover, through experimental results in MNIST we illustrate that our training method obtains strong performance with very limited training samples, outperforming the standard scheme for training Convolutional Neural Networks. Our empirical findings show that our model offers a compelling approach to prevent overfitting and circumscribe the generalization error, while at the same time the generative networks are able to produce samples with high perceptual score. The main caveat of our approach lies on the challenges of training in parallel multiple and competing neural networks.