deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets

Key Points

• The research aims to introduce and evaluate deltaBLEU as a metric for measuring the quality of generated text in diverse contexts.
• Developed deltaBLEU as a new discriminative metric for generation tasks.
• Compared deltaBLEU against traditional metrics on generation tasks with varied outputs.
• Utilized a benchmark dataset specific to natural language generation.
• DeltaBLEU outperformed traditional metrics in capturing diversity in generated text.
• Statistically significant improvements noted in evaluation reliability with deltaBLEU (p<0.05).
• Confirmed greater alignment with human judgment on diverse outputs compared to existing metrics.