Learning Continuous Phrase Representations for Translation Modeling

This paper tackles the sparsity problem in estimating phrase translation probabilities by learning continuous phrase representa-tions, whose distributed nature enables the sharing of related phrases in their represen-tations. A pair of source and target phrases are projected into continuous-valued vec-tor representations in a low-dimensional latent space, where their translation score is computed by the distance between the pair in this new space. The projection is performed by a neural network whose weights are learned on parallel training data. Experimental evaluation has been performed on two WMT translation tasks. Our best result improves the performance of a state-of-the-art phrase-based statistical machine translation system trained on WMT 2012 French-English data by up to 1.3 BLEU points. 1