Decoder Integration and Expected BLEU Training for Recurrent Neural Network Language Models

Neural network language models are often trained by optimizing likelihood, but we would prefer to optimize for a task specific metric, such as BLEU in machine trans-lation. We show how a recurrent neural network language model can be optimized towards an expected BLEU loss instead of the usual cross-entropy criterion. Fur-thermore, we tackle the issue of directly integrating a recurrent network into first-pass decoding under an efficient approxi-mation. Our best results improve a phrase-based statistical machine translation sys-tem trained on WMT 2012 French-English data by up to 2.0 BLEU, and the expected BLEU objective improves over a cross-entropy trained model by up to 0.6 BLEU in a single reference setup. 1