Abstract Background With the increasing prevalence of inflammatory bowel disease (IBD) and its complex therapeutic management, evaluating innovative approaches has become essential. Large language models (LLM), such as ChatGPT-4.0, have experienced remarkable growth recently. While several studies have assessed LLMs across different specialties, their application in IBD management remains unexplored. This study compares therapeutic recommendations generated by ChatGPT-4.0 with those made by gastroenterologist IBD specialists, evaluating its potential applications and limitations in clinical practice. Methods This prospective observational study was conducted at a University Hospital between September 2024 and March 2025, including 47 cases of IBD patients requiring initiation or modification of biologic therapy. Standardized clinical vignettes were developed to provide both the IBD specialists and ChatGPT-4.0 with similar information concerning patients’ presentations. For each case, questions were addressed in a consistent sequence: the choice of the therapeutic class of the treatment, followed by the specific molecule in this class, and finally if the proposition is a monotherapy or combination therapy. The propositions were compared using the percentage of agreement and the Cohen’s kappa coefficient (κ). Results The agreement between the IBD specialists and ChatGPT-4.0 was 76.6% (κ = 0.677) for the therapeutic class and 44.7% (κ = 0.352) for specific molecule selection. Therapeutic class discordance occurred in 11 cases (23.4%). All discordant cases were reviewed by the gastroenterologists and deemed acceptable choices. Among these, seven second-line suggestions made by ChatGPT-4.0 aligned with the IBD specialists’ treatment decisions. Conclusion This observational prospective study highlights a promising area of ChatGPT-4.0 application in the therapeutic management of IBD. However, it is not suitable as a standalone or fully reliable resource for decision-making in healthcare. Future efforts should focus on developing tailored healthcare GPT that incorporates European medical guidelines and experts’ feedback to further improve LLM performance. Conflict of interest: Dr. Loncour, Helena: No conflict of interest Aoun, Jennifer: No conflicts Muls, Vinciane: No conflict of interest Hoyois, Alice: Sponsored invitation to congress: Abbvie
Loncour et al. (Thu,) studied this question.