Towards Automated Performance Assessment using Velocity-based Motion Quality Metrics

Acquiring proficiency in endovascular surgery requires a significant investment of time and resources, both from trainees who are developing skills and from experienced surgeons who must serve as evaluators. These experienced surgeons typically provide feedback to trainees using structured grading scales that offer a qualitative and subjective assessment of performance. To address these limitations, we previously established that spectral arc length (SPARC), a frequency-domain measure of movement smoothness, was a quantitative and objective indicator of surgical experience. Still, trainees have indicated that performance feedback based on SPARC is not intuitive or easily understandable. In this work, we evaluate the potential of alternative quantitative measures of endovascular tool navigation proficiency. One set of metrics is available from a commercial endovascular surgical simulator, and another set of metrics is derived from tool tip velocity profiles. Results indicate that average guidewire tip velocities and idle times (the amount of time the guidewire remains stationary) are significantly different across experience groups. In contrast, only one of the performance metrics currently implemented on the simulator shows significant differences across experience groups. Subsequent analysis showed that average velocity and idle time correlate strongly with SPARC for these tasks. These results support the potential of metrics based on tool tip velocity for real-time objective assessment of endovascular skill. Further, these metrics, which correlate strongly to movement smoothness, are likely to be easier for participants to interpret than feedback based on spectral arc length, which could positively effect training effectiveness.