Evaluation and performance assessment on a low-cost, low-fidelity cricothyroidotomy simulator

Purpose Cricothyroidotomy is a critical emergency procedure performed when standard methods of airway management fail. With the reported incidence as low as 2.3 out of 1000 intubation attempts, hands-on learning experience is limited. This is where simulation could facilitate skill acquisition. Unfortunately, commercial cricothyroidotomy simulators are expensive, which can hinder training material procurement, especially in lower-resource settings. To address accessibility hurdles, we designed a low-cost, low fidelity cricothyroidotomy simulator with materials accessible worldwide. This study’s objective was to generate validity evidence for our simulator’s use in formative assessment. Methods Messick’s validity framework was used. Novices and experts’ cricothyroidotomy performance on the simulator was videotaped and assessed by two trauma surgeons using standardized Procedure-Specific (PCL) and Error Checklists (ECL) to generate a final timed score. Novice and expert mean scores were compared using a 2-tailed Mann–Whitney U test. Raters’ Cronbach’s α for the PCL and ECL were calculated to measure reliability. Results A total of 26 participants were recruited, with 19 participants in the novice arm and seven in the expert arm. Experts successfully completed more tasks (PCL z = -2.52, p = 0.012), committed fewer errors (ECL z = -3.07, p = 0.002) and obtained better final scores (z = -3.21, p = 0.001) when compared to novices. Item consistency on the PCL and ECL demonstrated excellent internal consistency (PCL Cronbach’s α = 0.86 and ECL Cronbach’s α = 0.85). Conclusions The findings provide preliminary validity evidence for the checklist-based assessment of simulated cricothyroidotomy performance on our low-cost, low fidelity simulator. Further research is needed to determine its educational effectiveness.