What question did this study set out to answer?

This study aims to enhance the reliability of deep learning models for predicting ICU outcomes by integrating data from multiple sources.

May 21, 2026Open Access

Multi-source data integration through pooling and transfer learning improves generalizability and specialization of deep learning models for ICU mortality and length of stay prediction: a four-database external validation study

JAJerome AllynInserm MOMatthieu OliverInserm RCRaphaël CerveauInserm

Key Points

This study aims to enhance the reliability of deep learning models for predicting ICU outcomes by integrating data from multiple sources.
External validation across four BlendedICU databases, extracting data from nearly 20,000 unique patients per source.
Comparison of training configurations, including single-source and multi-source strategies like data pooling and transfer learning.
Performance assessment using metrics such as AUROC, AUPRC, and MAPE for length of stay.
Data pooling yielded an Mcomposite improvement of up to 8.0% over single-source training models.
MAPE for length of stay with pooled data was 84.49 compared to 89.24 for transfer learning, with non-overlapping confidence intervals.
Internal validation overestimated external performance, with MAPE increasing by 51.8% and AUROC decreasing by 13.8% on external databases.

Abstract

BACKGROUND: Most Artificial Intelligence prognostic models in intensive care are trained and validated on a single source, limiting their reliability in external settings. This study evaluated generalizability and specialization for ICU mortality and remaining length of stay (RLoS) prediction across international databases, and explored multi-source strategies to mitigate the performance cost of external deployment. METHODS: Using the Temporal Pointwise Convolution (TPC) architecture, we conducted external validation across four harmonized BlendedICU databases (eICU-CRD, MIMIC-IV, AmsterdamUMCdb, HiRID), extracting nearly 20,000 unique patients from each. Four training configurations were compared: single-source training on AmsterdamUMC (N = 3,574) or MIMIC-IV (N = 10,915), data pooling, and transfer learning (both N = 14,489). Performance was assessed using AUROC and AUPRC for mortality, MAPE for RLoS, and a composite metric (Mcomposite). RESULTS: Internal validation consistently overestimated external performance, with MAPE increasing by up to 51.8% and AUROC dropping by up to 13.8% on external databases. Site-specific features such as drug exposure acted as "shortcuts" degrading portability. Data pooling emerged as the superior strategy, achieving the best generalization (Mcomposite improvement up to 8.0% over single-source models) while matching or outperforming transfer learning for specialization - particularly for RLoS (MAPE 84.49 vs. 89.24, non-overlapping confidence intervals). CONCLUSION: Single-source training compromises clinical applicability. Data pooling across harmonized international cohorts is the most effective strategy for both generalization and specialization, supporting a "Data-Centric AI" approach. We advocate for the democratization of local ICU datasets and open-source customizable models to empower independent clinical validation.

Ask AI

Helpful

Bookmark

View Full Paper