The Development of the Engagement Scale for Simulation-Based Education: Item Generation and Early Psychometric Findings

Background Simulation-based education (SBE) is an essential teaching modality in healthcare; learner engagement in SBE is critical for knowledge retention. Numerous scales are validated to measure engagement, but none are specific for use in SBE. We present development and initial validation evidence for the Engagement Scale for Simulation-Based Education (ESSBE), a self-report tool measuring behavioral, cognitive, emotional, and social dimensions of the engagement construct within scenario-based SBE. Methods A mixed methods approach included item generation, expert review, and cognitive pretesting. Learners already attending undergraduate and graduate healthcare education programs at a southeastern simulation center voluntarily completed an 18-item, 7-point frequency response questionnaire immediately after a simulation session (n = 402). Reliability was tested using Cronbach's alpha and Spearman's Rank (rho). Dimensionality was assessed via confirmatory factor analysis (CFA) and metric invariance testing. Results From several structural models tested, best-fit was obtained using a 2nd order/4-dimensional model with 11 retained items (CMIN/3.3, CFI/0.945, TLI/0.91, SRMR/0.055, RMSEA/0.075). Additional empirical evidence supported scale reliability (alphas ≥ 0.610; rho r(401) ≥ 0.434, P < 0.01) and metric invariance (held for Δ SRMR and Δ RMSEA). Conclusion Future studies are needed to strengthen the validation evidence for the ESSBE before the tool is ready for general use. For example, we have not yet tested convergent/divergent validity with existing metrics or whether ESSBE responses are associated with learner outcomes. We provide initial evidence supporting the 11-item, 4-dimension ESSBE as a potentially reliable and valid measurement of engagement in SBE.