Models and measures of learning outcomes for non-technical skills in simulation-based medical education: Findings from an integrated scoping review of research and content analysis of curricular learning objectives

Simulations are increasingly used to enhance ‘non-technical’ skills (NTS) in medical education but little is known about the effectiveness of many simulations. The field lacks robust outcome measures and there is lack of clarity about educationally relevant conceptual and operational dimensions of NTS to support instrument development and testing. Our study addresses this dual gap through a multi-component design focusing on three key NTS: interprofessional teamwork, communication and decision-making. A Scoping Review (ScR) was utilised to synthesise instruments used in recent publications on medical/clinical simulations targeting NTS. Full-text analysis identified a sub-set of robust instruments for further analysis. Systematic qualitative content analysis of learning objectives (LOs) in UK medical education curricula identified and synthesised the conceptual dimensions underlying NTS. Finally, identified measures’ dimensions were systematically compared with the dimensions of curricular LOs to establish the extent to which instruments used in current SBL-research address desired learning goals, and to identify relevant measures and gaps. The ScR identified 225 studies from 2018−20 of which 72 met the inclusion criteria. 31/72 studies referred to a named instrument in their abstract, including 27 unique instruments. The curriculum analysis identified a set of key conceptual dimensions of each of the target NTS. Finally, comparative analysis found that while the outcome measures used in the SBL-studies addressed many of the identified curricular LO-dimensions, there are significant gaps, notably relating to evidence use and inclusive practice. Moreover, it revealed there is little conceptual overlap between the instruments, highlighting the need for further research to ensure comparability of different studies. This study contributes to our understanding and evaluation of learning outcomes in, and our readiness to generate a rigorous evidence base for, NTS-focused SBL, by describing conceptual and operational constructs of NTS-learning outcomes in medical education and identifying appropriate validated assessment instruments to evaluate SBL.