Learning Evaluation Models From Large Language Models for Sequence Generation

Automatic evaluation of sequence generation, which has traditionally relied on metrics such as BLEU and ROUGE, often struggles to capture the semantic accuracy of generated text due to an overemphasis on n-gram overlap. A promising solution to this issue is the development of model-based metrics, such as BLEURT and COMET. However, these approaches are typically limited by the scarcity of labeled evaluation data, which is essential for training evaluation models. In this work, we address this challenge by proposing the Customized Sequence Evaluation Metric (CSEM), a three-stage model training method that leverages large language models to generate labeled data for metric development, eliminating the need for human-labeled data. Furthermore, we extend the capabilities of CSEM to support a range of evaluation types, including single-aspect, multi-aspect, reference-free, and reference-based evaluations. This flexibility allows for the customization of metrics to fit various real-world scenarios. Experimental results on the SummEval benchmark demonstrate that CSEM can effectively train an evaluation model without human-labeled data. Additional experiments in reinforcement learning and reranking show that metrics developed through CSEM outperform traditional evaluation metrics, leading to significant improvements in sequence quality, as assessed by both commonly used metrics and ChatGPT.