e16502 Background: Accurate postoperative risk stratification in localized clear-cell renal cell carcinoma (ccRCC) informs prognosis, surveillance intensity, and eligibility for adjuvant systemic therapy. Although validated prognostic models (Leibovich 2018, SSIGN, and UISS) exist, they are inconsistently applied in routine practice due to workflow and documentation barriers. Using AI assisted tools within the electronic medical record (EMR) may enable rapid, standardized risk assessment and improve clinical use. Methods: We retrospectively analyzed 92 patients with localized ccRCC undergoing radical nephrectomy and postoperative oncology evaluation (December 2021–September 2025) across three Mayo Clinic sites. Pathology reports and oncology notes were extracted via structured SQL queries. Clinical and pathologic variables were parsed using a zero-shot GPT-4o pipeline. Gold-standard prognostic risk classification and adjuvant therapy eligibility were established by an expert oncology clinician using validated prognostic calculators and KEYNOTE-564 eligibility criteria. AI-generated outputs and novice manual abstraction by two first-year fellows were compared against expert assessment using accuracy, precision, recall, and F1 score. Leibovich and SSIGN were evaluated as three-class models (low/intermediate/high), and UISS as binary risk stratification (high vs non-high). Results: Among 92 patients with localized ccRCC (median age 64 years range 41–78), AI-generated outputs demonstrated near-perfect agreement with expert assessment for KEYNOTE-564 adjuvant immunotherapy eligibility (accuracy 97%, precision 100%, recall 97%, F1 score 98%). For three-class prognostic stratification, AI achieved perfect agreement for Leibovich disease-free survival risk (accuracy, precision, recall, and F1 all 100%), outperforming novice abstraction. For SSIGN, AI demonstrated strong multi-class performance (accuracy 91%, precision 91%, recall 93%, F1 score 92%), compared with novice abstraction (accuracy 92%, F1 score 90%). For binary UISS risk stratification, AI achieved 81% accuracy, precision 74%, recall 100%, and F1 score 85% showing a tendency towards false positives. Conclusions: This study demonstrates AI can use existing prognostic and systemic therapy eligibility to accurately risk stratify and identify patients eligible for adjuvant systemic immunotherapy post nephrectomy. For our next steps, we plan to develop a multimodal AI platform integrating pathologic, radiologic, and clinical data to improve prognostication for outcomes and predictive capabilities for patients who would benefit from adjuvant systemic therapy for more personalized medicine.
Barker et al. (Thu,) studied this question.