What question did this study set out to answer?

February 20, 2026ComputersOpen Access

Rethinking Distractor Quality in Multimodal Multiple-Choice Questions: Automated Evaluation and Hard Benchmark Construction

View Full Paper

Ask AI

Bookmark

Authors

Automated metrics enhance distractor quality in multimodal multiple-choice questions, suggesting improved evaluation methods.

The aim is to improve the evaluation of distractor quality in multimodal multiple-choice questions using automated metrics.
Introduced 9 automated metrics to evaluate distractor quality.
Developed a metric-driven ensemble strategy to select optimal distractors.
Conducted evaluations with 33 multimodal large language models across 16 benchmarks.
The automated metrics reliably quantified distractor quality.
Generated distractors exhibited significantly higher confusability.
Presented a more rigorous challenge to existing state-of-the-art models.

Ding et al. (2026) studied this question.

View Full Paper

Ask AI

Bookmark

February 20, 2026ComputersOpen Access

View Full Paper

Ask AI

Bookmark

Authors

Automated metrics enhance distractor quality in multimodal multiple-choice questions, suggesting improved evaluation methods.

The aim is to improve the evaluation of distractor quality in multimodal multiple-choice questions using automated metrics.
Introduced 9 automated metrics to evaluate distractor quality.
Developed a metric-driven ensemble strategy to select optimal distractors.
Conducted evaluations with 33 multimodal large language models across 16 benchmarks.
The automated metrics reliably quantified distractor quality.
Generated distractors exhibited significantly higher confusability.
Presented a more rigorous challenge to existing state-of-the-art models.

Ding et al. (2026) studied this question.

View Full Paper

Ask AI

Bookmark