Endobronchial Ultrasound Skills and Tasks Assessment Tool: Assessing the Validity Evidence for a Test of Endobronchial Ultrasound-guided Transbronchial Needle Aspiration Operator Skill

RATIONALE: Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is becoming standard of care for the sampling of mediastinal adenopathy. The need for a safe, effective, accurate procedure makes EBUS-TBNA ideal for mastery training and testing. OBJECTIVES: The Endobronchial Ultrasound Skills and Tasks Assessment Tool (EBUS-STAT) was created as an objective competency-oriented assessment tool of EBUS-TBNA skills and knowledge. This study demonstrates the reliability and validity evidence of this tool. METHODS: The EBUS-STAT objectively scores the EBUS-TBNA operator's skills, including atraumatic airway introduction and navigation, ultrasound image acquisition and optimization, identification of mediastinal nodal and vascular structures, EBUS-TBNA sampling, and recognition of EBUS/computed tomography images of mediastinal structures. It can be administered at the bedside or using combination of low- and high-fidelity simulation platforms. Two independent testers administered the EBUS-STAT to 24 operators at three levels of EBUS-TBNA experience (8 beginners, 8 intermediates, and 8 experienced) at three institutions; operators were also asked to self-assess their skills. Scores were analyzed for intertester reliability, correlation with prior EBUS-TBNA experience, and association with self-assessments. MEASUREMENTS AND MAIN RESULTS: Intertester reliability between testers was very high (r = 0.9991, P < 0.00005). Mean EBUS-STAT scores for beginner, intermediate, and experienced groups, respectively, were 31.1, 74.9, and 93.6 out of 100 (F(2,21) = 118.6, P < 0.0001). Groups were nonoverlapping: post hoc tests showed each group differed significantly from the others (P < 0.001). Self-assessments corresponded closely to actual EBUS-STAT scores (r(2) = 0.81, P < 0.001). CONCLUSIONS: The EBUS-STAT can be used to reliably and objectively score and classify EBUS-TBNA operators from novice to expert. Its use to assess and document the acquisition of knowledge and skill is a step toward the goal of mastery training in EBUS-TBNA.