MTP3.04 Comparing Large Language Models in Predicting Multi-Disciplinary Team Meeting Outcomes

Abstract Aim Multidisciplinary team meetings (MDT) are an essential part of modern cancer care, involving multiple specialists discussing patient diagnosis and management. Increased workloads and complexity of patients and treatment put pressure on these meetings, worsened by clinical commitment pressures. Large language models (LLMs) can understand large volumes of information and respond to specific queries and prompts, and are beginning to be used as adjuncts to clinical care. We aim to assess the difference between two LLMS, Claude Opus 3 and Gemini, in predicting MDT outcomes. Methods A prospective diagnostic concordance and validation study, assessing the LLM’s ability to interpret clinical information and provide guideline-based management recommendations. We will provide the LLMs with the same clinical information that will be presented at MDT and prompt them to create treatment plans based Association of Breast Surgery and National Institute of Health and Care Excellence guidelines. The treatments plans will be reviewed and given a concordance score against the actual MDT outcome, Results Initial results demonstrate that both LLMs produce structured treatment plans according to the guidelines. We would predict that there is good concordance with the MDT outcome but that Claude is able to provide more reasoning regarding justified decision making. Conclusions We predict that Claude Opus 3, a more advanced LLM trained with professional data including medical data, would have better concordance with actual MDT outcomes. We predict it will also be able to understand more complex prompts including the need for further investigation and imaging.