Clinical expertise and cognitive reflection predict fewer misconceptions in Bayesian diagnostic reasoning

Background For many years, research has consistently confirmed that even medical experts fail to solve Bayesian diagnostic tasks. These tasks require correctly estimating a diagnostic test’s positive predictive value based on sensitivity, specificity, and the disease’s prevalence. Previous research has primarily focused on improving task presentation, for example by using natural frequencies or visual aids. The present study introduces the Medical Bayesian Performance Score (MBPS), a confidence-weighted measure that distinguishes confidently held misconceptions from missing concepts. Participants completed eight medical Bayesian diagnostic tasks and rated their confidence after each response. Using this approach, we examined whether medical knowledge, clinical expertise, and cognitive reflection are associated with performance in Bayesian diagnostic reasoning. Methods In an online survey, we compared MBPS values via t tests along two independent variables: (a) medical knowledge (physician group: N p = 30 vs. student control group: N s = 56) and (b) clinical expertise within the physician group (specialists and senior physicians: N e = 19 vs. resident physicians: N r = 11). Furthermore, we analyzed correlations between MBPS values and Cognitive Reflection Test (CRT) scores. Results Physicians did not achieve significantly higher MBPS values than the student control group. In contrast, specialists and senior physicians scored significantly higher than resident physicians. Across all groups, average MBPS values were negative, indicating that confidence-weighted incorrect responses outweighed confidence-weighted correct responses, a pattern consistent with misconceptions. Furthermore, higher CRT scores correlated significantly with higher MBPS values. Conclusions The MBPS provides a complementary perspective to conventional accuracy measures by distinguishing misconceptions from missing concepts and linking Bayesian diagnostic reasoning research with conceptual change theory. Consistent with this perspective, our findings suggest that formal medical knowledge alone was not associated with fewer misconceptions, whereas greater clinical expertise and higher cognitive reflection were. Future research should evaluate evidence-based conceptual change interventions for medical education, such as specific refutation-based training, as a means of improving Bayesian diagnostic reasoning.