Limitations of large language models in medical applications

Dear Editor, We read with great interest the article by Cuthbert and Simpson [1], which evaluated Chat Generative Pre-trained Transformer’s (ChatGPT’s) ability to pass the Fellowship of the Royal College of Surgeons examination in Trauma and Orthopaedic Surgery. In light of the recent surge of positive literature advocating for the incorporation of large language models (LLMs) in medical practice and education [2, 3], it is equally important to highlight their shortcomings as well. In this letter, we aim to supplement the authors’ study with a brief discussion of LLMs’ technical constraints that hinder their implementation and use in clinical and educational environments. To understand LLMs’ limitations, it is essential to first discuss their mechanism of action. These models undergo training on vast quantities of textual data; in the case of ChatGPT (and its foundational GPT model), these sources include Wikipedia entries, web pages, and online book corpora. Through extensive training, LLMs develop the ability to produce human-like responses to natural inputs ranging from simple queries to complex instructions.