Event-driven dynamic ambulance dispatch: A transformer-based reinforcement learning approach with model explainability

The global healthcare strain from events such as the COVID-19 pandemic has intensified ambulance shortages, leading to prolonged patient waiting and increased mortality. Efficient ambulance dispatch becomes paramount and presents the backbone for providing timely patient care. This paper tackles the dynamic ambulance dispatch problem as a semi-Markov decision process (SMDP) in which dispatch decisions are made immediately after the arrival of a new patient call or an idle ambulance, with the objective of minimizing severity-weighted patient waiting. To account for the dynamic and event-dependent decision-making, an elaborate characterization of the system transition dynamics is developed, in which strategic ambulance withholding is allowed. We propose a solution approach to the SMDP formulation by integrating a Transformer-based deep bipartite assignment with a reinforcement learning (RL) framework. This framework entails customized inter-agent (inter-ambulance) and inter-task (inter-patient) communications grounded in self-attention mechanisms, enabling effective contextual information exchange across ambulances and patients. In addition, model explainability is enabled by distilling the learned policies into a surrogate decision tree with theoretically grounded and human-understandable features. Computational experiments on both synthetic and real-world cases are conducted, with the results showing significantly reduced patient waiting and improved ambulance operational efficiency. Overall, the proposed approach contributes to the development of RL-based and explainable ambulance dispatching and has the potential to be applied to other settings of high-stakes dynamic service provision.