Why the study?
The 299-patient Heart Failure Clinical Records dataset is widely used for mortality prediction, yet reported performance varies substantially. Differences in prediction time, follow-up handling, resampling, and data splitting may materially affect apparent model performance.
Systematic Review (n=299)
No
In heart failure mortality prediction modeling, improper inclusion of follow-up duration (temporal leakage) and variations in preprocessing boundaries artificially inflate discrimination metrics, and models with similar AUROCs can still exhibit poor probability calibration.
Answers are grounded in this paper and cite their sources.
Parsimonious two-feature models may match complex ones for heart failure mortality prediction; leaves open calibration and leakage effects in benchmarks.
Background The 299-patient Heart Failure Clinical Records dataset is widely used for mortality prediction, yet reported performance varies substantially. Differences in prediction time, follow-up handling, resampling, and data splitting may materially affect apparent model performance. Objective To examine how prediction-time alignment, modelling procedures, and evaluation boundaries influence performance estimates on this shared benchmark, using a targeted methodological audit and harmonized reanalysis of representative modelling configurations. Methods Four representative modelling routes were evaluated using the same dataset. Admission-time classification used 11 baseline variables and stratified 10-fold cross-validation, with applicable resampling and feature-construction procedures restricted to the training data. A parsimonious random forest using ejection fraction and serum creatinine was also assessed. Follow-up duration was excluded from baseline classification and retained for time-to-event analyses. Sensitivity analyses examined follow-up-duration inclusion and changes in preprocessing boundaries. Patient-level bootstrap confidence intervals, repeated-seed analyses, calibration, and exploratory decision-curve analysis were additionally performed. Results Fold-mean AUROCs across the four 11-variable classification configurations ranged from 0.724 to 0.784. The two-feature random forest achieved a pooled out-of-fold AUROC of 0.779 (95% CI, 0.721–0.832), with a paired difference of 0.007 (95% CI, −0.047 to 0.062) from the 11-variable random-forest comparator. Adding follow-up duration increased fold-mean AUROC by 0.121–0.135 within matched predictor sets. The two-feature model retained substantial discrimination but showed limited calibration, with a calibration slope of 0.234 (95% CI, 0.154–0.404). Preprocessing sensitivity analyses also produced substantial variation in apparent performance. Conclusions Prediction-time alignment and evaluation procedures materially influenced performance estimates on this benchmark. Similar discrimination did not imply comparable probability calibration or equivalent patient-level risk estimates. Clinical interpretation therefore requires explicit definition of prediction time, outcome horizon, model implementation, and evaluation procedures, followed by validation in independent contemporary cohorts.
No takes yet. Share an insight, caveat, or question.
刁一娜 et al. (2026) conducted a systematic review in Heart Failure (n=299). Two-feature random forest mortality prediction model (ejection fraction and serum creatinine) vs. 11-variable random forest mortality prediction model was evaluated on Recorded death discrimination (pooled out-of-fold AUROC) (MD 0.007, 95% CI -0.047 to 0.062).
Synapse has enriched 10 related papers. Consider them for comparative context: