Training Phrase-Based Machine Translation Models on the Cloud: Open Source Machine Translation Toolkit Chaski

Training Phrase-Based Machine Translation Models on the Cloud: Open Source Machine Translation Toolkit Chaski In this paper we present an opensource machine translation toolkit Chaski which is capable of training phrase-based machine translation models on Hadoop clusters. The toolkit provides a full training pipeline including distributed word alignment, word clustering and phrase extraction. The toolkit also provides an extended error-tolerance mechanism over standard Hadoop error-tolerance framework. The paper will describe the underlying methodology and the design of the system, together with instructions of how to run the system on Hadoop clusters.