An empirical study of machine learning robustness and scalability for imbalanced tabular clinical data in emergency and critical care

Abstract Every year, millions of patients pass through emergency departments and intensive care units where clinicians must make life-altering decisions under time pressure and uncertainty. Advances in machine learning is poised to offer support for clinical decision-making, including prediction of patient deterioration, triage guidance, and identification of rare but clinically critical outcomes. Yet a persistent impediment limits its utilization in these settings: clinical data are often severely imbalanced, with critical outcomes occurring far less frequently than routine ones. This skewness can bias models toward majority classes, degrading performance. Developing models that are both robust to such imbalance and computationally efficient enough for deployment in time-sensitive environments remains an open and practically important challenge. In this paper, we empirically studied the robustness and scalability of six model families spanning classical machine learning, deep learning, and tabular foundation models on imbalanced tabular data from two large-scale clinical datasets (MIMIC-IV-ED and eICU). Class imbalance was quantified using three complementary metrics, and we compared tree-based methods (Decision Tree, Random Forest, XGBoost), the TabNet deep learning model, and two tabular foundation models (TabICL and TabPFN v2.6). All trainable models were evaluated under a unified experimental protocol using Bayesian hyperparameter optimization for trainable models, while foundation models were assessed in their pretrained inference regime without task-specific optimization or reweighting. All models were assessed on predictive performance (Macro F1-score), robustness to increasing imbalance, and computational scalability across seven clinically relevant prediction tasks. Results differed across databases. On MIMIC-IV-ED, foundation-based models (TabPFN v2.6 and TabICL) attained the strongest average Macro F1-score ranks, with XGBoost and other tree-based ensembles remaining competitive. On eICU, XGBoost consistently led, followed by other tree-based methods, while foundation models occupied intermediate positions. Across both datasets, TabNet exhibited the sharpest performance degradation under increasing imbalance and the highest computational costs. Training time analyses showed that classical and tree-based methods scale most favorably with dataset size, while foundation models achieved low per-task cost through their inference-based paradigm. These findings indicate that model selection for imbalanced clinical tabular data is context-dependent: no single family dominated across both datasets and all tasks. Nonetheless, recent advances in tabular foundation models suggest a rapidly narrowing performance gap with strong classical baselines such as XGBoost, while offering a distinct computational profile characterized by low per-task adaptation cost. This efficiency–performance trade-off may become increasingly relevant for deployment in resource-constrained clinical environments. Rather than prescribing a universal solution, this work provides clinical stakeholders with an empirically grounded framework for navigating the trade-offs between predictive robustness, computational scalability, and clinical feasibility in high-stakes, time-sensitive care environments.