Measuring the Impact of Artificial Intelligence on Report Drafting Efficiency in Chest CT Interpretation: A Retrospective Analysis (Preprint)

Background: Advancements in AI, especially deep learning, have revolutionized medical image analysis, potentially enhancing radiologists’ efficiency. Rising medical image volumes increase radiologists’ workloads, risking diagnostic accuracy and patient outcomes. In China, physician shortages, driven by high demand and limited resources, highlight AI’s role in addressing these gaps. However, evidence on AI’s impact on radiological efficiency is scarce and contradictory. Objective: This study aims to investigate the impact of an artificial intelligence (AI) system on the diagnostic efficiency of radiologists, particularly focusing on the reporting time in the diagnosis of lung nodules in Computed Tomography (CT) scans. Methods: Utilizing a pretest-posttest design with non-equivalent comparison groups, this research analyzed 195,020 CT reports from Beijing Anzhen and Tsinghua Changgung Hospitals (2018-2023) to assess changes in reporting times before, right after, and a few years after the implementation of the AI system using a difference-in-difference methodology. Results: After the introduction of the AI system, an initial increase in reporting time was observed among radiologists, with an overall average rise of 1.120 minutes (95% CI 0.965 to 1.275). In the first year after the AI implementation, Beijing Anzhen and Tsinghua Changgung Hospitals witnessed an increase in reporting times by 0.391 minutes (95% CI 0.091 to 0.691) and 0.990 minutes (95% CI 0.739 to 1.241), respectively. However, a reduction of 1.481 minutes (95% CI -1.744 to -1.218) in reporting time was observed three years after the AI system’s implementation at Beijing Anzhen hospitals. The delayed improvement in diagnostic efficiency after the AI implementation was deemed due to radiologists requiring additional time to familiarize themselves with such AI systems. Conclusions: While the immediate integration of AI systems in radiology may result in increased reporting times during an initial adaptation phase, prolonged utilization demonstrates potential improvements in radiologists’ efficiency in the diagnostic process of lung nodules in CT scans.