What is the clinical evidence from this study?

Study design: Observational. Population: Adults (≥18 years) admitted under a general medicine service at a tertiary hospital in Australia (n=4135). Primary outcome: Unplanned 30-day hospital readmission (ROC-AUC 0.64 for logistic regression model using structured EMR data).

What does this research mean for the field?

Logistic regression using structured EMR data achieves the highest discrimination (ROC-AUC 0.64) for predicting 30-day unplanned hospital readmissions in general medicine patients, outperforming more complex machine learning and multimodal models. Novelty: ClaimNovelty.INCREMENTAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

February 28, 2026Open Access

Comparative Evaluation of Machine Learning Models Using Structured and Unstructured Clinical Data for Predicting Unplanned General Medicine Readmissions in a Tertiary Hospital in Australia

Key Result

Logistic regression using structured EMR data achieved the highest discrimination with ROC-AUC 0.64 for predicting 30-day unplanned hospital readmissions in adult general medicine patients at a tertiary Australian hospital.

Key Points

To evaluate the effectiveness of machine learning models using structured and unstructured clinical data in predicting unplanned hospital readmissions.
Conducted a retrospective cohort study with 4,135 general medicine admissions.
Developed models using only structured data, only unstructured text, and a multimodal approach.
Evaluated classical machine learning models like logistic regression and XGBoost on structured features.
Assessed model performance using metrics such as ROC-AUC, accuracy, and F1-score.
24.3% of admissions experienced unplanned readmissions.
Logistic regression achieved the highest ROC-AUC of 0.64 among classical models.
Structured deep learning models yielded a ROC-AUC of 0.62.
Unstructured text-only and multimodal models had ROC-AUCs of 0.52 and 0.58 respectively.
Prior hospitalizations and comorbidity were significant predictors of readmissions.

Study Design

Type

Observational (n=4,135)

Multicenter

Structured PICO

Does incorporating unstructured clinical text into machine learning models improve the prediction of unplanned 30-day hospital readmissions compared to structured data alone in general medicine admissions?

Population

4135 general medicine admissions to a tertiary Australian hospital between July 2022 and June 2023.

Intervention

Deep learning models using unstructured text alone and a combined multimodal architecture integrating both structured and unstructured data.

Comparator

Models using structured data alone (deep learning and classical machine learning models including logistic regression, XGBoost, random forest, gradient boosting, extra trees, and HistGradient Boosting).

Outcome

Unplanned 30-day hospital readmissions.

Structured electronic medical record variables remain the primary drivers of 30-day readmission risk, and complex multimodal machine learning models do not outperform simple logistic regression for overall discrimination.

Main Result

Effect estimate: ROC-AUC 0.64 for logistic regression model using structured EMR data

Absolute Event Rate: 24.3% vs 75.7%

Limitations

Retrospective observational design limiting causal inference
Single-center study limiting generalizability
Exclusion of patients missing Hospital Frailty Risk Score reducing sample size
Modest overall predictive performance with AUCs below 0.7
Unstructured clinical text contributed limited improvement to prediction accuracy
Limited sensitivity in several models for identifying readmitted patients

Abstract

Background: Unplanned 30-day hospital readmissions, a key healthcare quality metric, are common and costly. Prediction models built on structured data often perform modestly, and the added value of unstructured clinical notes remains unclear. Methods: This retrospective cohort study included 4135 general medicine admissions to a tertiary Australian hospital between July 2022 and June 2023. Structured predictors included demographics, comorbidities, frailty, prior healthcare utilisation, length-of-stay, inflammatory markers, socioeconomic indicators, and lifestyle factors. We developed deep learning models using structured data alone, unstructured text alone, and a combined multimodal architecture integrating both modalities. For benchmarking, multiple classical machine learning models trained on structured features were evaluated using identical data splits, including logistic regression, XGBoost, random forest, gradient boosting, extra trees, and HistGradient Boosting. Model performance was assessed on a hold-out test set using ROC-AUC, accuracy, precision, recall, and F1-score. Results: Unplanned readmissions occurred in 24.3% of admissions. Among classical machine learning models, logistic regression achieved the highest discrimination (ROC-AUC 0.64), with no substantial improvement observed from ensemble methods. Structured-only deep learning achieved ROC-AUC 0.62. Unstructured text-only and multimodal models achieved ROC-AUCs of 0.52 and 0.58, respectively. Although overall discrimination of the multimodal model was lower than structured-only performance, it demonstrated improved sensitivity and F1-score for identifying patients who were readmitted. Prior hospitalisations, emergency department visits, and comorbidity burden were the strongest predictors. Conclusions: Structured EMR variables remain the main drivers of 30-day readmission risk. More complex classical machine learning models did not outperform logistic regression, and incorporating unstructured clinical text provided only modest improvement in identifying high-risk patients without enhancing overall discrimination.

Bookmark

View Full Paper