Context is everything: a context-aware annotation typology for dialogue translation quality assessment

Until recently, most machine translation (MT) systems translated sentences in isolation, neglecting crucial document-level context due to limited discourse-focused training data and a lack of robust evaluation methods. We introduce a context-aware annotation framework, validated on a customer support dataset with substantial inter-annotator agreement (Cohen’s κ = 0.73), potentially offering a new standard for contextual MT assessment.