Key result
Large language models classify free-text patient-reported symptoms post-surgery with a ~93% F1 score.
Why the study?
Existing electronic patient-reported outcome assessments with free text reporting are not well-equipped to capture emergent symptoms or nuanced concerns post-ambulatory surgery due to resource constraints.
Can large language models efficiently classify verbatim free text patient-reported symptoms post-ambulatory surgery for cancer?
Observational (n=1,070)
Can large language models efficiently classify verbatim free text patient-reported symptoms post-ambulatory surgery for cancer?
Large language models can accurately classify patient-reported symptoms from free text post-ambulatory surgery, offering a feasible pathway for real-time remote symptom monitoring.
Answers are grounded in this paper and cite their sources.
May aid AI symptom detection post-thyroidectomy; leaves open prospective validation before clinical adoption.
e18003 Background: As ambulatory procedures become commonplace in oncology, post-surgical remote symptom monitoring (RSM) is important for timely responses to emergent problems. Patient-reported outcomes (PROs) are the gold standard for the capture of the patient symptomatic experience; however, existing electronic PRO (ePRO) assessments that include free text reporting are not well-equipped to capture emergent symptoms or nuanced concerns due to resource constraints. Recent significant advances in AI modeling may allow for a minimally burdensome opportunity to monitor patients’ recovery. As a proof of concept, we aimed to determine whether large language models (LLMs) could efficiently classify verbatim free text patient-reported symptoms post-ambulatory surgery. Methods: Secure enterprise cloud-based versions of GPT-5.2 and Claude Sonnet 4.5, licensed to our institution, were used to identify symptoms from verbatim free-text responses collected from a retrospective sample of N = 1,070 cancer patients post-ambulatory surgical procedures for up to 10 days. Responses from thyroidectomy patients (n = 147) were coded and verified by a surgeon into 161 unique symptoms (e.g., ‘headache’, ‘dizziness’, ‘swollen incision’) and used as the gold standard. Coded responses were incorporated into RAG for postulated performance boost. The fine-tuned model was then used to analyze responses from mastectomy (n = 252) and prostatectomy (n = 671) patients. Results: GPT-5.2 with RAG identified 360 symptoms post-thyroidectomy, including (ordered by prevalence) ‘headache’ (area under the receiver operating characteristic curve [AUC]=0.89, 95% Confidence Interval [CI]: 0.83, 0.94), ‘dizziness’ (AUC=0.72, 95% CI: 0.72), ‘cough’ (AUC=0.81, 95% CI: 0.73, 0.90), ‘cough’ (AUC=0.99, 95% CI: 0.99, 1.00), and ‘sore throat’ (AUC=0.78, 95% CI: 0.65, 0.88). Patients differed in words chosen (e.g., ‘pain on swallowing’, ‘burning with swallowing’, and ‘difficulty swallowing’). Semantic similarity incorporated into modeling yielded a taxonomy of 98 symptom categories and an overall performance at 0.93 precision, 0.92 recall, and 0.93 F1 score. Claude Sonnet 4.5 had comparable results. Removing RAG resulted in a negligible reduction in model performance. In mastectomy and prostatectomy respectively, the most prevalent symptoms were ‘swelling’, ‘pain’, ‘drainage’, and ‘numbness’; and ‘swelling’, ‘pain’, ‘hematuria’, and ‘bleeding.’ Conclusions: Real-time processing of patient-reported free text post-ambulatory surgical concerns may be feasible using AI. RAG did not confer an advantage, due in part to the limited number of human-annotated symptoms. If further validated, AI-assisted RSM can offer patients an efficient and unfiltered pathway for reporting their post-surgical experiences in a way that can be integrated into clinical decision-making.
No takes yet. Share an insight, caveat, or question.
Cracchiolo et al. (2026) conducted an observational in Cancer post-ambulatory surgery (n=1,070). Large language models (GPT-5.2 and Claude Sonnet 4.5) with RAG vs. Surgeon-verified coded responses (gold standard) was evaluated on Symptom classification performance (F1 score 0.93). Large language models successfully classified free-text patient-reported symptoms post-ambulatory cancer surgery, achieving an overall precision of 0.93, recall of 0.92, and F1 score of 0.93.
Synapse has enriched 10 related papers. Consider them for comparative context: