What question did this study set out to answer?

This evaluation aims to assess the quality of TranslateGemma across multiple languages using MQM criteria.

March 18, 2026Open Access

MQM Quality Evaluation of TranslateGemma

View Full Paper

Ask AI

Bookmark

This evaluation aims to assess the quality of TranslateGemma across multiple languages using MQM criteria.
Conducted an MQM evaluation on TranslateGemma with 45 linguists annotating 322 segments.
Annotated translations included 16 target languages; 12 officially supported and 4 unsupported.
Measured errors across three severity levels and analyzed inter-annotator agreement.
Found a 6.2-fold quality gap between supported (95.7 penalty) and unsupported languages (15.3 penalty).
Moroccan Arabic outperformed 10 out of 12 supported languages in quality metrics.
MetricX-24 XXL showed the highest correlation with human evaluations (r=0.88).

No takes yet. Share an insight, caveat, or question.

Evaluation quantifies translation quality gaps in supported and unsupported languages, indicating substantial variances.

Alexander Murauski (2026) studied this question.

View Full Paper

Ask AI

Bookmark