Multimodal vision-language models for automated and explainable postoperative complication risk stratification

Postoperative patient deterioration remains a major challenge in intensive care, often going undetected because of alert fatigue and the cognitive burden of integrating diverse clinical data streams. Vital signs and narrative clinical notes are typically reviewed separately, requiring clinicians to manually connect numeric trends with textual observations under time pressure. Existing predictive models generally rely either on structured physiological data alone or on clinical text without grounding in concurrent physiological dynamics, while many lack transparent, evidence-linked outputs suitable for clinical deployment. To address this gap, we propose GROVE (GROunded Vision–languagE risk model for ICU), a multimodal vision-language framework for early postoperative risk detection. GROVE provides explainable and clinically actionable assessments by jointly modeling quantitative physiological signals and qualitative narrative information during the first 48 h following surgery using MIMIC-IV ICU data. The framework integrates: (1) structured physiological measurements, including heart rate, mean arterial pressure, respiratory rate, oxygen saturation, and temperature, represented as temporal plot images; and (2) unstructured nursing and provider notes. Unlike conventional classifiers, GROVE generates structured natural language assessments linking physiological trends with supporting clinical observations. Evaluation on the MIMIC-IV postoperative ICU cohort shows that GROVE achieves an AUROC of 0.92 for early detection of complications such as sepsis and hemorrhage, while 94% of generated outputs are rated evidence-aligned under clinician adjudication. By bridging quantitative and qualitative patient signals within an interpretable multimodal framework, GROVE reduces clinician cognitive burden and supports timely intervention, providing a scalable foundation for predictive and actionable clinical decision support.