Background: Large language models (LLMs) are emerging as clinical decision-support tools in oncology, yet their ability to generate reliable treatment recommendations in real-world multidisciplinary tumor board (MTB) settings remains uncertain, particularly for complex colorectal cancer (CRC). Methods: In this retrospective study, 300 consecutive adult CRC cases discussed at a tertiary MTB were evaluated. Standardized de-identified case summaries were independently submitted to Gemini 2.5 (general-purpose, guideline-integrated) and MedGemma 27B (domain-specialized; T = 0.0 and T = 1.0). Concordance with MTB decisions was assessed using weighted Cohen’s kappa (κ), accuracy, F1 score, and recall. Safety was adjudicated by blinded senior MTB members using a three-tier risk framework. Importantly, Gemini 2.5 was evaluated in a guideline-integrated setting, whereas MedGemma operated without external guideline retrieval, introducing a predefined asymmetry in knowledge augmentation. Results: Gemini 2.5 demonstrated substantial agreement (κ = 0.792, p < 0.001), the highest exact concordance (85.0%), and a clinical safety rate of 83.7%, significantly outperforming MedGemma (p < 0.001). MedGemma showed moderate agreement at T = 0.0 (κ = 0.566; accuracy = 70.3%) and modest improvement at T = 1.0 (κ = 0.610; accuracy = 73.5%; p = 0.038), with lower safety rates (62.3% and 62.7%; p < 0.001). Discordance predominantly occurred in clinically complex scenarios, including surgical/interventional planning, active surveillance, recurrent disease management, and patients with compromised performance status. Conclusions: A guideline-integrated general-purpose LLM demonstrated superior concordance and safety compared with a domain-specialized model operating without external retrieval, supporting its adjunctive use within MTBs while preserving expert clinical judgment.
Farzaliyeva et al. (Tue,) studied this question.