ChatGPT4 Performance on German CME - friend or foe (trick or treat)? (Preprint)

BACKGROUND The increasing development and spread of artificial and assistive intelligence is opening up new areas of application not only in applied medicine but also in related fields such as continuing medical education (CME), which is part of the mandatory training program for medical doctors in Germany. OBJECTIVE The aim of our study was to determine whether medical laypersons are able to successfully conduct training courses specifically for physicians with the help of a large language model such as ChatGPT4. METHODS We plan to conduct a randomized controlled trial in which high school students use ChatGPT4 to complete special training courses for doctors in the fields of internal medicine, surgery, gynecology, pediatrics, neurology and anesthesiology. Thus, the test is set up with three arms: a) input of full-text and CME questions in ChatGPT4, b) input of CME questions only and c) a solution approach using a keyword search function in the full text. By means of randomization, the participants were evenly distributed among the three arms. The trial was approved by the Ethics Committee of Witten/Herdecke University (No. S-108/2024, date of approval 15 May 2024) and registered on the Open Science Framework (https://doi.org/10.17605/OSF. IO/MZNUF) in advance. RESULTS not yet applicable. CONCLUSIONS Using this approach, we wanted to test further possible applications of artificial intelligence in the postgraduate medical education setting and obtain results for practical use. Depending on the results, the potential influence of LLMs such as Chat-GPT4 on CME will be discussed, e.g., as part of a SWOT analysis (strengths, weaknesses, opportunities, threats). CLINICALTRIAL Open Science Framework (https://doi.org/10.17605/OSF. IO/MZNUF)