B80-4-03 Evaluation of Sequential AI Risk Scoring for Malignancy Prediction in Indeterminate Lung Nodules

Abstract Rationale Artificial Intelligence (AI) models utilizing CT imaging have shown promise in estimating risk of malignancy for pulmonary nodules; however, most are trained at a single time point in screening cohorts. The performance and longitudinal behavior of these models on patients with indeterminate nodules followed with sequential imaging is unclear. We externally validated a deep learning lung nodule risk prediction model (RADLogics Inc, New York, NY) and evaluated whether change in AI risk score across follow-up imaging added predictive value for malignancy prediction. Methods Patients with indeterminate lung nodules 6-30mm in size referred to an institutional lung nodule clinic and followed with sequential imaging were labeled as having benign or malignant nodules based on histopathology or clinical diagnosis of benign nodules with ≥2 years of radiographic follow-up. The final two CT scans before diagnosis were analyzed using the AI software to derive a malignancy risk index (RMI). Patients were excluded if the interval between CT scans was less than 90 days. Binary classifier performance of the model was assessed at the initial and follow-up scan. Logistic regression was used to assess the relative contributions of the initial score (RMI) and the change in score (ΔRMI) in follow-up. Independent significance of ΔRMI was evaluated with the Wald test. Results A total of n = 61 patients with a 36% rate of malignancy were included. The median IQR time between initial and follow-up scans was 329 167-426 days. On the first scan, model AUC was 0.77 with PRC 0.63; at F1-optimized threshold the model had sensitivity 0.81, specificity 0.62, PPV 0.55, and F1 score of 0.65. On follow-up scan, model AUC was 0.89 with PRC 0.81 (Figure). On the follow-up scan, at F1-optimized threshold, the model had a sensitivity of 0.68, specificity of 0.92, PPV of 0.83, and F1 of 0.75. In logistic regression, each 0.1 increase in baseline RMI was associated with OR for malignancy of 2.92 (95% CI 1.69-5.03, p = 0.0001) and each 0.1 point ΔRMI between scans conferred OR 2.82 (95 % CI 1.57-5.09, p = 0.0005) and remained independently significant (Wald p = 0.0005) after adjustment for baseline RMI. Conclusions The AI-derived malignancy risk index demonstrated strong discriminative performance in a referred lung nodule cohort, including at timepoints before clinical practice pursued biopsy. Longitudinal changes in model-derived risk provided additional independent predictive information beyond baseline values. Trending AI-based malignancy scores may offer a useful adjunct to radiologic and clinical follow-up strategies for indeterminate pulmonary nodules. This abstract is funded by: None