How Thoroughly Are We Evaluating Disaster Management Trainings? A Scoping Review

Introduction: Given the rare occurrence of disasters and the complexities of managing them, it is crucial to ensure that responders receive comprehensive training. Maintaining high standards in disaster management (DM) training is essential for effective DM. However, there is a lack of literature on the monitoring and evaluation (M&E) of DM training programs. This review aimed to document the current M&E strategies used in DM training. Methods: On June 28, 2023, a systematic literature search was performed across the PubMed, Scopus, Embase, and Cochrane databases to identify studies that outlined the learning objectives and M&E strategies for DM training. The learning objectives were classified according to the revised Bloom’s Taxonomy, and the evaluation methodologies were aligned with the New World Kirkpatrick model. Results: 57 articles meet the inclusion and exclusion criteria, monitoring being reported by 5 studies, while all reported evaluation methods. The evaluations were centred around the second level of the New World Kirkpatrick model(N=57), with only 7 articles investigating the third level. 16 authors used existing M&E frameworks. When correlating the learning objectives with the evaluation methodology, a mismatch emerged, skills like the ability to “Apply” and “Create” being evaluated using the second level of the New World Kirkpatrick model, instead of the third one. Conclusion: The lack of monitoring and the low usage of existing M&E frameworks highlight a lack of awareness and standardization in the field. The mismatch between the learning objectives and the evaluation process may result in graduates being deemed ready to deploy despite facing hardships in real-world settings, potentially leading to unprepared responders.