We present a human judgements dataset and also an adapted metric for evalua-tion of Arabic machine translation. Our medium-scale dataset is first of its kind for Arabic with high annotation quality. We use the dataset to adapt the BLEU score for Arabic. Our score (AL-BLEU) provides partial credits for stem and mor-phological matchings of hypothesis and reference words. We evaluate BLEU, METEOR and AL-BLEU on our human judgements corpus and show that AL-BLEU has the highest correlation with hu-man judgements. We are releasing the The dataset and the software to the research community.
No takes yet. Share an insight, caveat, or question.
Bouamor et al. (2014) studied this question.