Large language model-augmented offline reinforcement learning framework for sepsis management in critical care

Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection, making optimal management critical. Existing Reinforcement Learning (RL) approaches for sepsis management have mainly relied on structured data (e.g., vital signs, laboratory results), lacking contextual information needed for comprehensive patient understanding. In this work, we propose a Multimodal Offline REinforcement learning for Clinical notes Leveraged Enhanced stAte Representation (MORE-CLEAR) framework for sepsis management. MORE-CLEAR employs large language models (LLMs) to facilitate the extraction of rich semantic representations from clinical notes, preserving clinical context and improving patient state representation. Gated fusion and cross-modal attention allow dynamic weight adjustment and the effective integration of multimodal data. Cross-validation using two public (MIMIC-III, MIMIC-IV) and one tertiary ICU dataset (SNUH) showed that MORE-CLEAR significantly improved the estimated survival rate and policy performance compared to single-modal RL. This approach could expedite sepsis management by enabling RL models to propose effective actions.