Training generative neural networks via Maximum Mean Discrepancy optimization

We consider training a deep neural network to generate samples from an distribution given i. i. d. data. We frame learning as an optimization a two-sample test statistic---informally speaking, a good generator produces samples that cause a two-sample test to fail to reject the hypothesis. As our two-sample test statistic, we use an unbiased estimate the maximum mean discrepancy, which is the centerpiece of the nonparametric two-sample test proposed by Gretton et al. (2012). We compare to the nets framework introduced by Goodfellow et al. (2014), in which is a two-player game between a generator network and an adversarial network, both trained to outwit the other. From this perspective, MMD statistic plays the role of the discriminator. In addition to empirical, we prove bounds on the generalization error incurred by optimizing empirical MMD.