Uncertainty-calibrated adaptation of clinical transformer foundation models enhances in-hospital mortality and hospital readmission prediction

Accurate prediction of in-hospital mortality and 30-day readmission is important for clinical decision support, yet existing transformer approaches focus narrowly on ICU cohorts, rely on multimodal inputs that few institutions can harmonize, and rarely provide calibrated uncertainty suitable for triage. We address these gaps with a portability-first design, studying the broad general hospital admission population—spanning both ICU and general ward patients—using only ICD procedure code sequences combined with age and sex. This minimal, universally recorded input is itself a contribution, yielding a model substrate that is interoperable, auditable, and applicable to resource-constrained settings. On this foundation, we conduct the first systematic comparison of uncertainty-aware parameter-efficient adaptation strategies for clinical transformer foundation models on general admissions, evaluating full-parameter fine-tuning, Single LoRA, LoRA Ensemble, Bayesian LoRA, and Monte Carlo Dropout against classical baselines, with a leakage-aware mortality protocol on MIMIC-IV and external validation on MIMIC-III. Our analysis shows that no single uncertainty mechanism dominates: stochastic averaging on a fully fine-tuned backbone delivers the strongest discrimination, Bayesian adaptation is best calibrated on the highly imbalanced mortality outcome, and ensemble averaging is best calibrated on the more heterogeneous readmission outcome, while the clinical transformer generalizes more gracefully than classical baselines under domain shift. Together, these findings establish that procedure-code sequences alone, paired with uncertainty-calibrated parameter-efficient adaptation, can support reliable, deployment-ready risk estimates across the general hospital admission population, and clarify how to match the uncertainty mechanism to outcome prevalence and clinical objective.