Key points are not available for this paper at this time.
Background and purpose: 2023 has witnessed an explosion in artificial intelligence technology with the public release of large-language models such as ChatGPT (OpenAI). Early studies have shown encouraging results on ChatGPT performance in complex medical tasks, such as completing the United States Medical Licencing Exam (USMLE) and generating a scientific manuscript. Methods: We submitted ten case files of varying complexity (4 simple, 3 moderate, and 3 complex) to ChatGPT and asked it to recommend management. This was compared with the multidisciplinary team meeting recommendations. Results: Management algorithms were similar to multidisciplinary team meeting recommendations in 70% of cases. Surgical and radiation treatment recommendations were largely accurate, however medical oncology recommendations were less often correct. In complex cases, with multiple tumours or repeat surgery, ChatGPT was unable to correctly interpret the pathology reports and correctly summarise the case. Furthermore, ChatGPT was unable to recommend suitable clinical trials for the submitted case files. Conclusions: While ChatGPT shows promise in natural language interpretation and natural language response, there are significant limitations in response accuracy. Further work is required to optimise artificial intelligence for use in medical treatment decision making. References: 1. Sorin, V., Barash, Y., Konen, E. & Klang, E. Deep-learning natural language processing for oncological applications. Lancet Oncol. 21, 1553–1556 (2020). 2. Sorin, V., Barash Y., Konen E., Klang E. Large language models for oncological applications. J. Cancer Res. Clin. Oncol. https://doi.org/10.1007/s00432-023-04824-w.
Ng et al. (Sat,) studied this question.