Multidisciplinary teams (MDTs) are central to treatment planning for colorectal cancer liver metastases (CRCLM) but require time and consistent access to expertise. Chat-based large language models (LLMs) such as ChatGPT can generate recommendations from written clinical summaries; however, their concordance with MDT decisions in CRCLM is not well characterized. We conducted a single-center retrospective concordance study of 30 consecutive CRCLM cases discussed at an MDT. ChatGPT was provided a standardized anonymized text synopsis (without direct imaging access) and asked for management recommendations under two a priori conditions: (1) baseline synopsis only, and (2) a conditional query in which resectability status was explicitly specified. Each case and condition was queried three independent times in separate sessions using identical prompts; outputs were mapped to predefined management categories. Agreement between the final LLM recommendation and MDT decisions was assessed using percent agreement and Cohen’s kappa. Across repeated runs, the LLM assigned the same management category in all cases (within-model consistency 100%, 3/3) for both querying conditions. In the baseline condition, agreement with MDT decisions was 66.7% (20/30; Cohen’s kappa = 0.606, moderate agreement). In the conditional resectability-specified condition, agreement was 93.3% (28/30; Cohen’s kappa = 0.924, very good agreement). Baseline discordant cases were characterized by conservative model outputs, including recommendations for systemic therapy and/or additional diagnostic work-up; only two cases remained discordant after resectability was specified. A chat-based LLM showed moderate concordance with unanimous MDT recommendations from minimal case summaries and very good concordance when resectability status was explicitly specified. These findings support feasibility as a supervised decision-support adjunct, but do not establish clinical benefit; prospective outcome-based validation is required.
Yılmaz et al. (Wed,) studied this question.