Evaluating performance: a scoping review on video-based assessments of non-technical skills in the operating room

Background: Patient safety remains a critical concern in the high-stakes environment of the operating room (OR). Human factors and non-technical skills (NTS) play pivotal roles in surgical performance and in preventing errors. Various assessment tools and methodologies have been developed to evaluate NTS among OR personnel, reflecting the growing recognition of their importance. Objectives: To map and analyze NTS among OR personnel during real-life surgeries using medical video recording systems and to evaluate NTS assessment tools described in existing literature. Methods: Four databases, PubMed, Web of Science, Medline and Embase, were searched for relevant studies. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) protocol was followed. Eligibility criteria included studies focusing on video recordings providing an overview of the entire OR with a focus on NTS. Results: Twenty-four articles were included in this scoping review. A total of eight NTS were assessed, most frequently communication (79%), leadership (54%), teamwork (50%) and situational awareness (50%). Observational methods varied, ranging from one or more cameras to more sophisticated medical data recorders, such as the OR Black Box (ORBB). Additionally, a wide variety of assessment methods were used, mostly validated tools (NOTSS, SPLINTS, ANTS), but also broader systems-based frameworks (SEIPS) or self-developed tools. Conclusions: This scoping review highlights the diverse methodologies used to measure NTS in the OR using medical video recordings, indicating a need for further research to address challenges and standardize practices. Despite the proven potential benefits of video recording in the OR, legal, ethical and logistical challenges may serve as possible barriers to actual implementation.