The Potential of Large Language Models in Gynecologic Oncology Telephone Triage

Introduction: Nurse-led telephone triage, frequently guided by the United Kingdom Oncology Nursing Society (UKONS) tool, is a cornerstone of supportive oncology care, yet the appropriateness of triage decisions varies across settings. Large Language Models (LLMs) show promise in medical communication but remain unvalidated for autonomous clinical decision-making, particularly in gynecologic oncology, where remote and telephone-based care pose distinct challenges. Methods and Materials: Twenty fictitious case vignettes representing common treatment-related toxicities in gynecologic oncology were developed. Three board-certified gynecologic consultants independently assigned UKONS Red–Amber–Green (RAG) triage categories, reaching unanimous consensus that served as the reference standard. Vignettes were submitted verbatim, as single-turn queries with no follow-up prompting, to three publicly available LLMs (OpenEvidence, ChatGPT, Google Gemini). LLM responses were mapped to RAG categories and compared descriptively against the reference standard, stratified by treatment type and platform. Results: All three LLMs achieved 100% concordance with the reference standard for all Red cases across both treatment groups, with no high-acuity case downgraded to self-care. Discrepancies clustered at lower-acuity boundaries. In the chemotherapy-alone group, errors were limited to omission of an explicit 24-h follow-up window for 1 Amber-rated complaints. In the combined group, all LLMs over-triaged intermediate-risk complaints toward immediate or same-day evaluation; low-risk complaints showed more variable over-triage. Conclusions: LLMs identified unambiguous emergent gynecologic oncology symptoms, preserving the UKONS safety floor, but showed relatively reduced precision and a conservative over-triage bias at lower-acuity margins, especially in the combined-therapy collective. These findings support a supervised, adjunctive role for LLMs, rather than autonomous deployment, pending prospective validation.