Answers are grounded in this paper and cite their sources.
Background ICU prediction models must retain clinically meaningful performance across differences in patients, measurement practices and outcome recording. Discrimination alone cannot establish this reliability. We examined model transportability and prediction-set coverage under a common cross-database design. Methods In this retrospective study, adults still recorded in the index ICU stay at 24 hours were evaluated for subsequent recorded death before index ICU discharge. Four model families used six vital-sign concepts, common preprocessing and 27,437 training patients per source. Sixteen configurations per family underwent source-only screening followed by three-seed confirmation of two finalists. Mean source-validation AUROC determined selection before final test evaluation. Reciprocal testing, separate probability and conformal calibration, hospital-disjoint eICU partitions, overlap weighting and measurement thinning assessed complementary aspects of reliability. Results Cohorts comprised 54,376 MIMIC-IV and 102,581 eICU patients. Source validation selected a Transformer in both directions. Source-test and external AUROCs were 0.776 and 0.720 for MIMIC-IV development, and 0.767 and 0.738 for eICU development. MIMIC-IV-trained gradient boosting had higher external AUROC, 0.743. The selected models overestimated external mortality in one direction and underestimated it in the other. A domain classifier distinguished the databases with AUROC 0.957. Marginal prediction sets covered 88.9% and 91.1% of external outcomes, but only 1.4% and 0.0% of deaths. Class-conditional death coverage was 91.6% and 84.9%, with both labels included for 54.2% and 49.3% of patients. Separate target updating yielded death coverage of 88.9% and 91.4%, with two-label frequencies of 56.1% and 54.8%. Conclusions Common inputs and training budgets improved the interpretability of cross-database comparisons but did not establish measurement equivalence or clinical reliability. Source model selection, external probability accuracy and event-specific coverage require separate evaluation. Prediction-set coverage must be interpreted alongside informativeness and the target data required for updating.
No takes yet. Share an insight, caveat, or question.
Tianyi Yu (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: