Machine learning–based differentiation of malignant and non-malignant pleural effusion in lung cancer patients: a multicenter retrospective study

Malignant pleural effusion (MPE) is a common complication of lung cancer and is associated with poor prognosis and substantial clinical burden. Differentiating malignant from non-malignant pleural effusion remains challenging, particularly when definitive pleural fluid cytology or pleural biopsy results are pending, inconclusive, or delayed. This study aimed to develop and validate a machine learning–based model for auxiliary classification of MPE in lung cancer patients after lung cancer diagnosis and partial tumor-related pathological or staging information had been obtained. In this retrospective multicenter study, a total of 715 lung cancer patients with pleural effusion were included from two tertiary hospitals. The Shanxi cohort from Heping Hospital Affiliated to Changzhi Medical College included 420 patients and was used as the development cohort, whereas the Anhui cohort from Tongling People’s Hospital included 295 patients and served as the independent external validation cohort. The development cohort was randomly divided into a training set and an internal test set at a ratio of 7:3, resulting in 295 patients in the training set and 125 patients in the internal test set. Clinical, laboratory, and pathological variables were collected. Feature selection was performed using least absolute shrinkage and selection operator regression within the training set. Seven machine-learning algorithms were developed and evaluated. Model performance was assessed using receiver operating characteristic curves, calibration analysis, and decision curve analysis. External validation was conducted using the independent Anhui cohort. Six predictors were selected, including age, Ki-67 index, lymph node metastasis, tumor size, total cholesterol, and serum albumin. Multiple machine learning models demonstrated comparable performance in internal validation. The XGBoost model achieved an area under the curve (AUC) of 0.97 in the internal test set and 0.91 in the independent external validation cohort. At the training-derived operating threshold, the external validation sensitivity was 0.35 and the specificity was 0.98, indicating favorable overall discrimination but limited rule-out performance. SHAP analysis suggested that tumor burden–related variables and systemic condition had substantial influence on model output. A machine learning–based model was developed for auxiliary classification of malignant and non-malignant pleural effusion in lung cancer patients after lung cancer diagnosis and partial tumor-related pathological or staging information had been obtained. The model showed favorable discrimination but limited sensitivity at the training-derived operating threshold in external validation and should therefore be regarded only as a candidate auxiliary tool. Further prospective and multicenter validation and recalibration are required before clinical implementation, and the model is not intended to replace cytological or histopathological confirmation.