Establishment and validation of multiclassification prediction models for pulmonary nodules based on machine learning

Abstract Background Lung cancer is the leading cause of cancer‐related death worldwide. This study aimed to establish novel multiclassification prediction models based on machine learning (ML) to predict the probability of malignancy in pulmonary nodules (PNs) and to compare with three published models. Methods Nine hundred fourteen patients with PNs were collected from four medical institutions (A, B, C and D), which were organized into tables containing clinical features, radiologic features and laboratory test features. Patients were divided into benign lesion (BL), precursor lesion (PL) and malignant lesion (ML) groups according to pathological diagnosis. Approximately 80% of patients in A (total/male: 632/269, age: 57.73 ± 11.06) were randomly selected as a training set; the remaining 20% were used as an internal test set; and the patients in B (total/male: 94/53, age: 60.04 ± 11.22), C (total/male: 94/47, age: 59.30 ± 9.86) and D (total/male: 94/61, age: 62.0 ± 11.09) were used as an external validation set. Logical regression (LR), decision tree (DT), random forest (RF) and support vector machine (SVM) were used to establish prediction models. Finally, the Mayo model, Peking University People's Hospital (PKUPH) model and Brock model were externally validated in our patients. Results The AUC values of RF model for MLs, PLs and BLs were 0.80 (95% CI: 0.73–0.88), 0.90 (95% CI: 0.82–0.99) and 0.75 (95% CI: 0.67–0.88), respectively. The weighted average AUC value of the RF model for the external validation set was 0.71 (95% CI: 0.67–0.73), and its AUC values for MLs, PLs and BLs were 0.71 (95% CI: 0.68–0.79), 0.98 (95% CI: 0.88–1.07) and 0.68 (95% CI: 0.61–0.74), respectively. The AUC values of the Mayo model, PKUPH model and Brock model were 0.68 (95% CI: 0.62–0.74), 0.64 (95% CI: 0.58–0.70) and 0.57 (95% CI: 0.49–0.65), respectively. Conclusions The RF model performed best, and its predictive performance was better than that of the three published models, which may provide a new noninvasive method for the risk assessment of PNs.