What question did this study set out to answer?

The aim is to develop a framework for automated surgical skill assessment that is both high-performing and interpretable.

April 26, 2026Open Access

An interpretable whole-stream video based deep learning framework for surgical skill assessment of basic procedural elements

Key Points

The aim is to develop a framework for automated surgical skill assessment that is both high-performing and interpretable.
I3D-SAP framework uses a 3D convolutional neural network to analyze surgical videos.
Performance evaluation conducted on the JIGSAWS dataset focusing on suturing, needle passing, and knot tying.
Class Activation Map visuals provide interpretability and insight into the model's classification process.
Achieved 100% accuracy in suturing, 96.3% in needle passing, and 97.2% in knot tying.
Performance dropped significantly under the Leave-One-User-Out protocol, revealing generalization issues.
Testing on an external dataset showed near-chance-level accuracy, indicating a need for better surgeon-invariant feature learning.

Abstract

Background: Conventional surgical skill assessment, conducted by expert surgeons in operating rooms, is often time-consuming and subjective. An automated and objective system capable of differentiating surgeon expertise levels (novice, intermediate, and expert) and providing feedback would significantly enhance surgical training. However, most existing automated frameworks prioritize deep learning performance over interpretability. Materials and methods: We propose I3D-SAP, an interpretable video-based surgical skill assessment framework of basic procedural elements using a 3D convolutional neural network. The model processes surgical videos and classifies surgeons into three expertise levels. Experiments are conducted on the public JIGSAWS dataset, evaluating performance across suturing, needle passing, and knot tying tasks. Results: Our method has achieved high accuracy: 100% (suturing), 96.3% (needle passing), and 97.2% (knot tying), comparable to the state-of-the-art models. Our framework’s interpretability is demonstrated through Class Activation Map (CAM) visualization and snippet-wise analysis, offering dual perspectives to explain classification decisions. However, evaluation under a more rigorous Leave-One-User-Out (LOUO) protocol revealed a significant performance drop, and testing on an external dataset (ROSMA) resulted in near-chance-level accuracy, indicating that the current model’s ability to generalize to entirely unseen surgeons is limited. Conclusion: I3D-SAP provides both high accuracy and interpretability, addressing a critical gap in automated surgical assessment of basic procedural elements. By visualizing the model decisions, it enhances transparency and trust, paving the way for practical adoption in surgical training. Moreover, the model’s current generalizability challenges should be addressed through future work involving larger, more diverse datasets and techniques for surgeon-invariant feature learning before practical clinical deployment can be considered.

Read Full Paperexternally

AI에게 질문

Bookmark

View Full Paper