What question did this study set out to answer?

The study aims to examine how cognitive biases in prompts affect the accuracy of large language models on radiology exam questions.

Cognitively Biased Prompt Effects on Large Language Model Accuracy for Radiology Board-Style Examination Questions

View Full Paper

Ask AI

Bookmark

Authors

Investigates cognitive bias impacts on language model accuracy in radiology exams, suggesting significant vulnerabilities exist.

The study aims to examine how cognitive biases in prompts affect the accuracy of large language models on radiology exam questions.
Evaluated ten large language models on 400 examination-style questions.
Used three cognitive bias prompts: authority bias, complexity bias, and anchoring bias.
Implemented two mitigation strategies: a prompt bias audit and a one-shot mitigation approach.
Baseline accuracy: 84.8% for text-based and 59.5% for multimodal questions.
Cognitive bias prompts resulted in accuracy declines of up to 44.9% for multimodal questions.
Mitigation strategies improved accuracy by up to 24.9% for multimodal questions.

Dietrich et al. (2026) studied this question.

View Full Paper

Ask AI

Bookmark