Recent advances in reasoning-enhanced large language models (LLMs) and multimodal LLMs (MLLMs) have significantly improved performance in complex tasks, yet medical AI models often overlook the structured reasoning processes inherent in clinical practice. In this work, we propose a two-stage pipeline to train ChestX-Reasoner, a radiology diagnosis MLLM with process supervision mined directly from clinical reports, reflecting the step-by-step reasoning followed by radiologists. To facilitate and evaluate reasoning capabilities, we introduce RadRBench-CXR, a comprehensive benchmark featuring 59K visual question answering samples with 301K clinically validated reasoning steps, and RadRScore, a metric evaluating reasoning factuality, completeness, and effectiveness. Here we show that, ChestX-Reasoner achieves significant improvements of 16% and 8.5% in reasoning ability compared to leading medical and general-purpose models, respectively. Furthermore, diagnostic accuracy improves by 3.3% to 24% against baselines. These results demonstrate that incorporating explicit reasoning steps improves diagnostic outcomes, and integrating process supervision enhances the reliability and transparency of medical diagnosis. In this study, we developed ChestX-Reasoner, an artificial intelligence system for analyzing chest X-rays. To build this system, we used information from routine clinical reports so that it could learn to provide both answers and explanations. To validate our approach, we introduced a comprehensive benchmark, RadRBench-CXR, along with a specialized evaluation metric, RadRScore. Our findings demonstrate that ChestX-Reasoner significantly outperforms existing models in both reasoning quality and final answer accuracy, leading to more robust and transparent clinical decision-making. Fan et al., train a chest radiology multimodal large language model using a two stage pipeline with explicit, report derived reasoning steps and introduced large scale benchmarks to assess clinical reasoning. The model substantially improved reasoning and diagnostic accuracy, showing that process supervision boosts reliability and transparency.
Fan et al. (Wed,) studied this question.