Question Generation via Generative Adversarial Networks

Question Generation (QG) models suffer from exposure bias. Exposure bias refers to the problem that an autoregressive generator is trained given the observable preceding context, though it runs during test conditioned on the self-generated context. Exposure bias unavoidably causes performance degradation. In this paper, we enhance the GAN-based SeqGAN model which slightly contributes to alleviating exposure bias. Instead of RNN and CNN, we leverage UniLMv2 to construct the generator and discriminator in SeqGAN. Different from RNN who generates the current token in terms of the closely adjacent token, UniLMv2 depends on all the preceding contexts to predict the current token. This reduces the risk of severe token-by-token error propagation caused by exposure bias. During the representation learning process, we retain the adversarial mechanism of SeqGAN that drives the generation of human-like languages, as well as the rewarding mechanism which reduces the loss if subsequent tokens are perfectly generated. Though, we carry out a straight rewording using N-grams. The experiments on the benchmark dataset SQuAD show that our model outperforms the state-of-the-art QG models for the metrics BLEU-4, ROUGE-L and METEOR. Besides, our model is less time-consuming compared to the original SeqGAN. The time consumption of 2nd-stage learning is 1h47m on GeForce RTX 2080 Ti.